scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-04 02:31:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 71

Merge remote-tracking branch 'origin/main' into litellm_projects_nav_team_admins
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
Details
LiteLLM Rust / rust-test (push) Has been cancelled
Details
LiteLLM Rust / rust-wheel (push) Has been cancelled
Details
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Details
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Details
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Details
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Details

Browse source 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	ui/litellm-dashboard/src/app/(dashboard)/projects/_components/ProjectsPage.tsx
This commit is contained in:
Devin AI 2026-10-03 01:38:13 +00:00
commit da3f494201
92 changed files with 4100 additions and 2083 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
deploy/lens/README.md
View file

@ -19,11 +19,13 @@ The URL, database, and retention settings can also come from `CLICKHOUSE_URL`, `
Retention changes require a proxy restart. ClickHouse removes expired rows during background merges, not immediately at startup. Enable request/response logging to analyze LLM requests. Lens can only inspect content you actually retain
In Lens, click **Set up analysis**, choose an existing virtual key or **Create worker key**, then **Generate setup command**. The LiteLLM address is filled in for you; change it only if the server running Docker needs a different network address. Copy the command and run it on your server. The dialog changes to **Analyzer connected** when the container checks in
In **Lens > Investigations**, click **Connect worker**, choose an analysis model and monthly limit, then **Get install command**. Use **Advanced options** to select an existing virtual key or change the proxy URL if the server running Docker needs a different network address. Copy the command and run it on your server. The dashboard shows **Worker connected** when the container checks in
The command already contains the compatible worker image and one worker token. The selected virtual key stays on the proxy; its secret is never sent to the worker. No source checkout, environment file, or second LiteLLM deployment is needed. Keep the command private because it includes the token. The LiteLLM release provides the dashboard and APIs; the container only runs background analysis
The dashboard and Compose file pin a verified worker image by digest. The image uses Linux amd64, and the generated command selects that platform. Worker image releases are independent of proxy releases: update the pinned image when changing their API contract. CI also publishes immutable commit tags for reproducible builds
The dashboard and Compose file pin a verified worker image by digest. The image uses Linux amd64, and the generated command selects that platform. CI also publishes immutable `:sha-<commit>` tags for successful worker builds on `main`. Keep the worker image compatible with your gateway version
After upgrading the gateway, update the worker image and redeploy it while keeping its proxy URL and token. Existing containers do not update automatically. If an investigation reports a worker compatibility error, update the image before retrying
For deployments managed with Compose, download `compose.yaml` and provide `LITELLM_URL` and `LENS_WORKER_TOKEN` in an environment file. Its default image is already selected:

2
deploy/lens/compose.yaml
View file

@ -1,6 +1,6 @@
services:
lens-worker:
image: ${LENS_WORKER_IMAGE:-ghcr.io/berriai/litellm-lens-worker@sha256:67eba741c1b97c749975c5c38e2370a603e1105babc908d613c1b79d7b995393}
image: ${LENS_WORKER_IMAGE:-ghcr.io/berriai/litellm-lens-worker@sha256:44f0597c7583dcfef999ece9a8bc02cfeb9f0f5167a1221cee3bd10b1b79271b}
environment:
LITELLM_URL: ${LITELLM_URL:?Set the URL reachable from this container}
LENS_WORKER_TOKEN: ${LENS_WORKER_TOKEN:?Create a worker credential in the Lens UI}

2
litellm-rust/crates/storage-clickhouse/src/lib.rs
View file

@ -4,7 +4,7 @@ mod read;
pub use error::Error;
pub use insert::{insert_compressed_rows, insert_encoded_rows};
pub use read::{Parameter, Query, execute_read, fetch, fetch_json};
pub use read::{Parameter, Query, READ_LIMITS, ReadLimits, execute_read, fetch, fetch_json};
use url::Url;
#[derive(Clone)]

22
litellm-rust/crates/storage-clickhouse/src/read.rs
View file

@ -5,7 +5,18 @@ use serde::{Deserialize, Serialize, de::DeserializeOwned};
use crate::{Connection, Error};
const MAX_RESPONSE_BYTES: usize = 4 * 1024 * 1024;
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct ReadLimits {
pub result_rows: u64,
pub response_bytes: usize,
pub execution_seconds: u64,
}
pub const READ_LIMITS: ReadLimits = ReadLimits {
result_rows: 1000,
response_bytes: 4 * 1024 * 1024,
execution_seconds: 10,
};
#[derive(Debug, Deserialize, Serialize)]
#[serde(untagged)]
@ -78,9 +89,12 @@ pub async fn execute_read(
.clear()
.extend_pairs(existing_pairs)
.append_pair("readonly", "1")
.append_pair("max_result_rows", "1000")
.append_pair("max_result_rows", &READ_LIMITS.result_rows.to_string())
.append_pair("result_overflow_mode", "throw")
.append_pair("max_execution_time", "10")
.append_pair(
"max_execution_time",
&READ_LIMITS.execution_seconds.to_string(),
)
.append_pair("wait_end_of_query", "1")
.append_pair("default_format", "JSON");
@ -101,7 +115,7 @@ pub async fn execute_read(
let mut body = Vec::new();
while let Some(chunk) = response.chunk().await.map_err(|_| Error::Transport)? {
if body.len() + chunk.len() > MAX_RESPONSE_BYTES {
if body.len() + chunk.len() > READ_LIMITS.response_bytes {
return Err(Error::ResponseTooLarge);
}
body.extend_from_slice(&chunk);

2
litellm-rust/crates/traces-clickhouse/src/lib.rs
View file

@ -12,7 +12,7 @@ pub use error::Error;
pub use insert::{InsertRow, InsertTable, encode_rows, insert_rows, insert_shared_rows};
pub use litellm_storage_clickhouse::{Connection, Parameter};
pub use litellm_traces::{QueryScope, ReadQuery};
pub use query::{execute_read, query_help, query_sql};
pub use query::{QueryHelp, execute_read, query_help, query_sql};
pub use query_access::QueryReaders;
pub use schema::{
NORMALIZED_FIELD_DEFINITIONS, NormalizedFieldDefinition, ensure_schema, schema_statements,

292
litellm-rust/crates/traces-clickhouse/src/query.rs
View file

@ -6,11 +6,14 @@ use futures_util::{
stream::{self, TryStreamExt},
};
use litellm_http::Client;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use serde::{Deserialize, Serialize, Serializer};
use serde_json::Value;
use strum::IntoEnumIterator;
use super::{Connection, Error, NORMALIZED_FIELD_DEFINITIONS, Parameter};
use super::{
Connection, Error, NORMALIZED_FIELD_DEFINITIONS, NormalizedFieldDefinition, Parameter,
query_access::READER_LIMITS,
};
mod guide;
pub mod lens;
@ -48,10 +51,42 @@ enum PathPart {
Index(usize),
}
#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd, Serialize, strum::Display)]
#[serde(rename_all = "lowercase")]
#[strum(serialize_all = "lowercase")]
enum JsonKind {
Array,
Boolean,
Integer,
Null,
Number,
Object,
String,
}
impl JsonKind {
fn of(value: &Value) -> Self {
match value {
Value::Null => Self::Null,
Value::Bool(_) => Self::Boolean,
Value::Number(number) if number.is_i64() || number.is_u64() => Self::Integer,
Value::Number(_) => Self::Number,
Value::String(_) => Self::String,
Value::Array(_) => Self::Array,
Value::Object(_) => Self::Object,
}
}
}
#[derive(Clone, Copy, Debug, Serialize, strum::Display)]
enum MapValueType {
String,
}
#[derive(Serialize)]
struct MetadataField {
path: Vec<PathPart>,
types: BTreeSet<&'static str>,
types: BTreeSet<JsonKind>,
expression: String,
}
@ -66,42 +101,150 @@ struct ColumnSchema {
#[derive(Serialize)]
struct TableSchema {
name: &'static str,
name: TraceTable,
columns: Vec<ColumnSchema>,
}
trait Unobserved {
fn unobserved() -> Self;
}
enum Discovery<T> {
Observed(T),
Unavailable(String),
}
impl<T: Serialize + Unobserved> Serialize for Discovery<T> {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
#[derive(Serialize)]
struct Unavailable<'a, T> {
#[serde(flatten)]
sample: T,
error: &'a str,
}
match self {
Self::Observed(sample) => sample.serialize(serializer),
Self::Unavailable(error) => Unavailable {
sample: T::unobserved(),
error,
}
.serialize(serializer),
}
}
}
#[derive(Serialize)]
struct MetadataCatalog {
table: &'static str,
column: &'static str,
struct MetadataSample {
fields: Vec<MetadataField>,
sampled_rows: usize,
invalid_json_rows: usize,
truncated: bool,
}
impl Unobserved for MetadataSample {
fn unobserved() -> Self {
Self {
fields: Vec::new(),
sampled_rows: 0,
invalid_json_rows: 0,
truncated: true,
}
}
}
#[derive(Serialize)]
struct MetadataCatalog {
table: TraceTable,
column: &'static str,
#[serde(flatten)]
discovery: Discovery<MetadataSample>,
sample_sql: &'static str,
scope: &'static str,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
}
#[derive(Serialize)]
struct AttributeField {
key: String,
#[serde(rename = "type")]
kind: &'static str,
kind: MapValueType,
expression: String,
}
#[derive(Serialize)]
struct AttributeCatalog {
table: &'static str,
column: &'static str,
struct AttributeSample {
fields: Vec<AttributeField>,
truncated: bool,
}
impl Unobserved for AttributeSample {
fn unobserved() -> Self {
Self {
fields: Vec::new(),
truncated: true,
}
}
}
#[derive(Serialize)]
struct AttributeCatalog {
table: TraceTable,
column: &'static str,
#[serde(flatten)]
discovery: Discovery<AttributeSample>,
discovery_sql: String,
scope: &'static str,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
}
#[derive(Serialize)]
struct NormalizedField {
table: TraceTable,
name: &'static str,
column: &'static str,
#[serde(rename = "type")]
kind: &'static str,
meaning: &'static str,
}
impl From<&NormalizedFieldDefinition> for NormalizedField {
fn from(field: &NormalizedFieldDefinition) -> Self {
Self {
table: TraceTable::OtelTraces,
name: field.name,
column: field.clickhouse_column,
kind: field.clickhouse_type,
meaning: field.meaning,
}
}
}
#[derive(Serialize)]
struct Relationship {
left: &'static str,
right: &'static str,
additional_predicates: &'static str,
meaning: &'static str,
}
const RELATIONSHIPS: [Relationship; 1] = [Relationship {
left: "otel_traces.LiteLLMRequestId",
right: "spend_logs.response_id",
additional_predicates: "otel_traces.TeamId = spend_logs.team_id AND (otel_traces.TeamId != '' OR (otel_traces.UserId != '' AND otel_traces.UserId = spend_logs.user) OR (otel_traces.ApiKeyHash != '' AND otel_traces.ApiKeyHash = spend_logs.api_key))",
meaning: "The normalized ID is the response ID, not request_id. Cached requests can share response_id; joins may return multiple spend rows",
}];
#[derive(Serialize)]
pub struct QueryHelp {
dialect: &'static str,
access: &'static str,
response: &'static str,
tables: Vec<TableSchema>,
normalized_fields: Vec<NormalizedField>,
metadata: MetadataCatalog,
attributes: Vec<AttributeCatalog>,
relationships: &'static [Relationship],
examples: [guide::Example; 5],
gotchas: [String; 11],
guide: String,
}
pub async fn execute_read(
@ -153,22 +296,16 @@ fn metadata_expression(path: &[PathPart]) -> String {
fn discover(
value: &Value,
path: Vec<PathPart>,
fields: &mut BTreeMap<Vec<PathPart>, BTreeSet<&'static str>>,
fields: &mut BTreeMap<Vec<PathPart>, BTreeSet<JsonKind>>,
) -> bool {
if path.len() > MAX_DEPTH || (fields.len() >= MAX_FIELDS && !fields.contains_key(&path)) {
return true;
}
if !path.is_empty() {
let kind = match value {
Value::Null => "null",
Value::Bool(_) => "boolean",
Value::Number(number) if number.is_i64() || number.is_u64() => "integer",
Value::Number(_) => "number",
Value::String(_) => "string",
Value::Array(_) => "array",
Value::Object(_) => "object",
};
fields.entry(path.clone()).or_default().insert(kind);
fields
.entry(path.clone())
.or_default()
.insert(JsonKind::of(value));
}
match value {
Value::Object(object) => object.iter().fold(false, |limited, (key, value)| {
@ -194,7 +331,7 @@ fn discover(
}
}
fn metadata_catalog(sample: &[MetadataRow]) -> MetadataCatalog {
fn metadata_sample(sample: &[MetadataRow]) -> MetadataSample {
let (fields, limited, invalid_rows) = sample.iter().take(SAMPLE_ROWS).fold(
(BTreeMap::new(), sample.len() > SAMPLE_ROWS, 0),
|(fields, limited, invalid_rows), row| match serde_json::from_str::<Value>(&row.metadata) {
@ -214,24 +351,19 @@ fn metadata_catalog(sample: &[MetadataRow]) -> MetadataCatalog {
types,
})
.collect();
MetadataCatalog {
table: "spend_logs",
column: "metadata",
MetadataSample {
fields,
sampled_rows: sample.len().min(SAMPLE_ROWS),
invalid_json_rows: invalid_rows,
truncated: limited,
sample_sql: METADATA_SQL,
error: None,
scope: METADATA_SCOPE,
}
}
pub async fn query_help(client: &Client, connection: &Connection) -> Result<String, Error> {
pub async fn query_help(client: &Client, connection: &Connection) -> Result<QueryHelp, Error> {
let tables = stream::iter(TraceTable::iter())
.then(|table| async move {
Ok::<_, Error>(TableSchema {
name: table.into(),
name: table,
columns: rows::<ColumnSchema>(
client,
connection,
@ -242,13 +374,15 @@ pub async fn query_help(client: &Client, connection: &Connection) -> Result<Stri
})
.try_collect::<Vec<_>>()
.await?;
let metadata = match rows::<MetadataRow>(client, connection, METADATA_SQL).await {
Ok(sample) => metadata_catalog(&sample),
Err(error) => MetadataCatalog {
error: Some(error.to_string()),
truncated: true,
..metadata_catalog(&[])
let metadata = MetadataCatalog {
table: TraceTable::SpendLogs,
column: "metadata",
discovery: match rows::<MetadataRow>(client, connection, METADATA_SQL).await {
Ok(sample) => Discovery::Observed(metadata_sample(&sample)),
Err(error) => Discovery::Unavailable(error.to_string()),
},
sample_sql: METADATA_SQL,
scope: METADATA_SCOPE,
};
let attributes = stream::iter(["SpanAttributes", "ResourceAttributes"])
.then(|column| async move {
@ -257,27 +391,27 @@ pub async fn query_help(client: &Client, connection: &Connection) -> Result<Stri
(SELECT {column} FROM otel_traces WHERE Timestamp >= now() - INTERVAL 7 DAY \
LIMIT 200) ORDER BY key LIMIT 201"
);
let (keys, error) = match rows::<AttributeRow>(client, connection, &sql).await {
Ok(keys) => (keys, None),
Err(error) => (Vec::new(), Some(error.to_string())),
let discovery = match rows::<AttributeRow>(client, connection, &sql).await {
Ok(keys) => Discovery::Observed(AttributeSample {
truncated: keys.len() > MAX_FIELDS,
fields: keys
.into_iter()
.take(MAX_FIELDS)
.map(|row| AttributeField {
expression: format!("{column}[{}]", literal(&row.key)),
key: row.key,
kind: MapValueType::String,
})
.collect(),
}),
Err(error) => Discovery::Unavailable(error.to_string()),
};
let fields = keys
.iter()
.take(MAX_FIELDS)
.map(|row| AttributeField {
key: row.key.clone(),
kind: "String",
expression: format!("{column}[{}]", literal(&row.key)),
})
.collect();
AttributeCatalog {
table: "otel_traces",
table: TraceTable::OtelTraces,
column,
fields,
truncated: error.is_some() || keys.len() > MAX_FIELDS,
discovery,
discovery_sql: sql,
scope: ATTRIBUTE_SCOPE,
error,
}
})
.collect::<Vec<_>>()
@ -287,33 +421,31 @@ pub async fn query_help(client: &Client, connection: &Connection) -> Result<Stri
normalized_fields: &NORMALIZED_FIELD_DEFINITIONS,
metadata: &metadata,
attributes: &attributes,
limits: &READER_LIMITS,
};
Ok(json!({
"dialect": "ClickHouse SQL",
"access": "Request-log visibility enforced by ClickHouse row policies; proxy admins see all rows, users see their own rows and permitted teams, and callers without user identity see their own key rows",
"response": "ClickHouse JSON envelope: meta, data, rows, statistics; 64-bit integers may be strings",
"tables": tables,
"normalized_fields": NORMALIZED_FIELD_DEFINITIONS.iter().map(|field| json!({
"table": "otel_traces", "name": field.name, "column": field.clickhouse_column,
"type": field.clickhouse_type, "meaning": field.meaning
})).collect::<Vec<_>>(),
"metadata": metadata,
"attributes": attributes,
"relationships": [{
"left": "otel_traces.LiteLLMRequestId", "right": "spend_logs.response_id",
"additional_predicates": "otel_traces.TeamId = spend_logs.team_id AND (otel_traces.TeamId != '' OR (otel_traces.UserId != '' AND otel_traces.UserId = spend_logs.user) OR (otel_traces.ApiKeyHash != '' AND otel_traces.ApiKeyHash = spend_logs.api_key))",
"meaning": "The normalized ID is the response ID, not request_id. Cached requests can share response_id; joins may return multiple spend rows"
}],
"examples": guide.examples()?,
"gotchas": guide.gotchas()?,
"guide": guide::render(&guide)?,
}).to_string())
Ok(QueryHelp {
dialect: "ClickHouse SQL",
access: "Request-log visibility enforced by ClickHouse row policies; proxy admins see all rows, users see their own rows and permitted teams, and callers without user identity see their own key rows",
response: "ClickHouse JSON envelope: meta, data, rows, statistics; 64-bit integers may be strings",
examples: guide.examples()?,
gotchas: guide.gotchas()?,
guide: guide::render(&guide)?,
normalized_fields: NORMALIZED_FIELD_DEFINITIONS
.iter()
.map(NormalizedField::from)
.collect(),
relationships: &RELATIONSHIPS,
tables,
metadata,
attributes,
})
}
#[cfg(test)]
mod tests {
use super::*;
use rstest::rstest;
use serde_json::json;
#[rstest]
fn metadata_discovery_preserves_mixed_types_and_reports_invalid_rows() {
@ -328,7 +460,7 @@ mod tests {
metadata: "invalid".into(),
},
];
let catalog = json!(metadata_catalog(&sample));
let catalog = json!(metadata_sample(&sample));
assert_eq!(
catalog["fields"],
json!([{
@ -351,7 +483,7 @@ mod tests {
metadata: json!(metadata).to_string(),
})
.collect();
let catalog = json!(metadata_catalog(&sample));
let catalog = json!(metadata_sample(&sample));
assert_eq!(catalog["truncated"], true);
assert_eq!(catalog["sampled_rows"], row_count.min(SAMPLE_ROWS));
assert_eq!(

5
litellm-rust/crates/traces-clickhouse/src/query/guide.rs
View file

@ -1,8 +1,8 @@
use askama::Template;
use serde::Serialize;
use super::{AttributeCatalog, MetadataCatalog, TableSchema};
use crate::{Error, NormalizedFieldDefinition};
use super::{AttributeCatalog, Discovery, MetadataCatalog, TableSchema};
use crate::{Error, NormalizedFieldDefinition, query_access::ReaderLimits};
#[derive(Template)]
#[template(path = "query_help.jinja", escape = "none", blocks = [
@ -33,6 +33,7 @@ pub(super) struct QueryGuide<'a> {
pub normalized_fields: &'a [NormalizedFieldDefinition],
pub metadata: &'a MetadataCatalog,
pub attributes: &'a [AttributeCatalog],
pub limits: &'a ReaderLimits,
}
#[derive(Serialize)]

40
litellm-rust/crates/traces-clickhouse/src/query_access.rs
View file

@ -2,6 +2,7 @@ use std::{sync::Arc, time::Duration};
use hmac::{Hmac, Mac};
use litellm_http::Client;
use litellm_storage_clickhouse::READ_LIMITS;
use litellm_traces::QueryScope;
use moka::future::Cache;
use strum::IntoEnumIterator;
@ -11,6 +12,33 @@ use tokio::sync::{OwnedSemaphorePermit, Semaphore};
use super::{Connection, Error, TraceTable};
const MIB: u64 = 1024 * 1024;
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(crate) struct ReaderLimits {
pub result_rows: u64,
pub result_bytes: u64,
pub memory_bytes: u64,
pub execution_seconds: u64,
}
impl ReaderLimits {
pub fn result_mib(&self) -> u64 {
self.result_bytes / MIB
}
pub fn memory_mib(&self) -> u64 {
self.memory_bytes / MIB
}
}
pub(crate) const READER_LIMITS: ReaderLimits = ReaderLimits {
result_rows: READ_LIMITS.result_rows,
result_bytes: READ_LIMITS.response_bytes as u64,
memory_bytes: 256 * MIB,
execution_seconds: READ_LIMITS.execution_seconds,
};
#[derive(Clone)]
pub struct QueryReaders {
writer: Connection,
@ -75,13 +103,19 @@ impl QueryReaders {
return Err(Error::InvalidScope);
}
let password_hash = format!("{:x}", Sha256::digest(password));
let ReaderLimits {
result_rows,
result_bytes,
memory_bytes,
execution_seconds,
} = READER_LIMITS;
self.execute(
client,
format!(
"CREATE USER IF NOT EXISTS {user} IDENTIFIED WITH sha256_hash BY '{password_hash}' \
SETTINGS readonly = 1 CONST, max_execution_time = 10 CONST, \
max_result_rows = 1000 CONST, max_result_bytes = 4194304 CONST, \
result_overflow_mode = 'throw' CONST, max_memory_usage = 268435456 CONST, \
SETTINGS readonly = 1 CONST, max_execution_time = {execution_seconds} CONST, \
max_result_rows = {result_rows} CONST, max_result_bytes = {result_bytes} CONST, \
result_overflow_mode = 'throw' CONST, max_memory_usage = {memory_bytes} CONST, \
max_threads = 2 CONST, max_concurrent_queries_for_user = 8 CONST"
),
)

10
litellm-rust/crates/traces-clickhouse/src/table.rs
View file

@ -1,6 +1,14 @@
#[derive(
Clone, Copy, Debug, strum::Display, strum::AsRefStr, strum::EnumIter, strum::IntoStaticStr,
Clone,
Copy,
Debug,
serde::Serialize,
strum::Display,
strum::AsRefStr,
strum::EnumIter,
strum::IntoStaticStr,
)]
#[serde(rename_all = "snake_case")]
#[strum(serialize_all = "snake_case")]
pub enum TraceTable {
OtelTraces,

20
litellm-rust/crates/traces-clickhouse/templates/query_help.jinja
View file

@ -11,18 +11,18 @@ Normalized span fields
{% endfor %}
Observed LLM call metadata
{{ metadata.scope }}
Sampled rows: {{ metadata.sampled_rows }}; invalid JSON rows: {{ metadata.invalid_json_rows }}; truncated: {{ metadata.truncated }}
{% if let Some(error) = metadata.error %}Metadata discovery unavailable: {{ error }}
{% else if metadata.fields.is_empty() %}No metadata paths found in the sampled rows
{% else %}{% for field in metadata.fields %}{{ field.expression }}: {% for kind in field.types %}{{ kind }} {% endfor %}
{% endfor %}{% endif %}
{% match metadata.discovery %}{% when Discovery::Unavailable(error) %}Metadata discovery unavailable: {{ error }}
{% when Discovery::Observed(sample) %}Sampled rows: {{ sample.sampled_rows }}; invalid JSON rows: {{ sample.invalid_json_rows }}; truncated: {{ sample.truncated }}
{% if sample.fields.is_empty() %}No metadata paths found in the sampled rows
{% else %}{% for field in sample.fields %}{{ field.expression }}: {% for kind in field.types %}{{ kind }} {% endfor %}
{% endfor %}{% endif %}{% endmatch %}
Observed span and resource attributes
{% for catalog in attributes %}{{ catalog.table }}.{{ catalog.column }}
{{ catalog.scope }}
{% if let Some(error) = catalog.error %}Attribute discovery unavailable: {{ error }}
{% else if catalog.fields.is_empty() %}No attribute keys found in the sampled spans
{% else %}{% for field in catalog.fields %}{{ field.expression }}: {{ field.kind }}
{% endfor %}{% endif %}{% endfor %}
{% match catalog.discovery %}{% when Discovery::Unavailable(error) %}Attribute discovery unavailable: {{ error }}
{% when Discovery::Observed(sample) %}{% if sample.fields.is_empty() %}No attribute keys found in the sampled spans
{% else %}{% for field in sample.fields %}{{ field.expression }}: {{ field.kind }}
{% endfor %}{% endif %}{% endmatch %}{% endfor %}
Examples
{% block recent_spans_name %}Recent normalized LLM spans{% endblock %}
@ -44,7 +44,7 @@ Gotchas
{% block time_window %}Always bound Timestamp or start_time and use LIMIT; add TeamId/ApiKeyHash or team_id/api_key filters when investigating one tenant{% endblock %}
{% block reader_limits %}The reader enforces 1000 result rows, 4 MiB response bytes, 256 MiB memory and a 10 second query limit; exceeding limits fails instead of returning partial results{% endblock %}
{% block reader_limits %}The reader enforces {{ limits.result_rows }} result rows, {{ limits.result_mib() }} MiB response bytes, {{ limits.memory_mib() }} MiB memory and a {{ limits.execution_seconds }} second query limit; exceeding limits fails instead of returning partial results{% endblock %}
{% block reader_profile %}LiteLLM provisions SELECT-only readers from the configured ClickHouse connection and enforces request-log visibility through row policies. Callers see their own user rows and permitted teams, or their own key rows when no user identity is available. Provisioning requires CREATE USER, ALTER USER, CREATE ROW POLICY, and GRANT SELECT permissions{% endblock %}

8
litellm-rust/crates/traces-clickhouse/tests/migrations.rs
View file

@ -1467,8 +1467,8 @@ async fn query_help_discovers_live_schema_and_runs_its_examples(
)
.await?;
}
let help: serde_json::Value = serde_json::from_str(
&litellm_traces_clickhouse::query_help(&database.client, &reader).await?,
let help = serde_json::to_value(
litellm_traces_clickhouse::query_help(&database.client, &reader).await?,
)?;
let keys: std::collections::BTreeSet<_> = help
.as_object()
@ -1654,8 +1654,8 @@ async fn query_help_preserves_schema_and_guide_when_discovery_hits_reader_limits
}))).collect::<Result<Vec<_>, _>>()?;
insert_rows(&database, "otel_traces", spans).await?;
let reader = Connection::configured(&database.url, "trace_test", "help_reader", "")?;
let help: serde_json::Value = serde_json::from_str(
&litellm_traces_clickhouse::query_help(&database.client, &reader).await?,
let help = serde_json::to_value(
litellm_traces_clickhouse::query_help(&database.client, &reader).await?,
)?;
assert_eq!(help["tables"].as_array().ok_or("tables")?.len(), 3);
assert!(!help["examples"].as_array().ok_or("examples")?.is_empty());

2
litellm-rust/crates/traces-clickhouse/tests/query_access.rs
View file

@ -94,7 +94,7 @@ async fn queries_and_help_are_scoped_by_the_database(
.await?,
)?;
assert_eq!(summary["data"][0]["count"], json!(expected.len()));
let help = query_help(&database.client, &reader).await?;
let help = serde_json::to_string(&query_help(&database.client, &reader).await?)?;
assert_eq!(help.contains("secret_b"), expected.contains(&"b"));
assert_eq!(help.contains("secret-b"), expected.contains(&"b"));
let recreated = QueryReaders::new(database.writer.clone(), "trace_test".to_owned());

4
litellm-rust/crates/traces/tests/query/named.rs
View file

@ -46,10 +46,10 @@ fn named_requests_preserve_all_access_cases(#[case] teams: Vec<&str>, #[case] ke
#[rstest]
fn result_contracts_preserve_public_field_names() {
round_trip::<ListTracesRow>(
json!({"trace_id": "trace", "trace_ref": "ref", "team_id": "team", "api_key_hash": "key", "user_id": "user", "name": "agent", "service": "service", "input_preview": "input", "status": "ok", "start_ms": -1, "duration_ms": 20, "span_count": u64::MAX, "agent_count": 1, "agent_invocations": 2, "llm_calls": 3, "tool_calls": 4, "input_tokens": 5, "output_tokens": 6, "models": ["model"], "error_count": 0, "request_ids": ["request"]}),
json!({"trace_id": "trace", "trace_ref": "ref", "team_id": "team", "api_key_hash": "key", "user_id": "user", "name": "agent", "service": "service", "input_preview": "input", "status": "ok", "start_ms": -1, "duration_ms": 20, "span_count": u64::MAX, "agent_count": 1, "agent_invocations": 2, "agent_names": ["agent"], "frameworks": ["framework"], "llm_calls": 3, "tool_calls": 4, "input_tokens": 5, "output_tokens": 6, "models": ["model"], "error_count": 0, "request_ids": ["request"]}),
);
round_trip::<TraceSpansRow>(
json!({"span_id": "span", "parent_span_id": "parent", "name": "agent", "type": "agent", "agent": "agent", "status": "error", "status_message": "error", "error_truncated": 1, "start_ns": -1, "duration_ns": u64::MAX, "service": "service", "input_preview": "input", "model": "model", "input_tokens": u32::MAX, "output_tokens": 6, "litellm_request_id": "request", "team_id": "team", "api_key_hash": "key", "user_id": "user"}),
json!({"span_id": "span", "parent_span_id": "parent", "name": "agent", "type": "agent", "agent": "agent", "framework": "framework", "status": "error", "status_message": "error", "error_truncated": 1, "start_ns": -1, "duration_ns": u64::MAX, "service": "service", "input_preview": "input", "model": "model", "input_tokens": u32::MAX, "output_tokens": 6, "litellm_request_id": "request", "team_id": "team", "api_key_hash": "key", "user_id": "user"}),
);
round_trip::<SpanDetailRow>(
json!({"span_id": "span", "input": "input", "output": "output", "attributes": {"count": "42"}}),

128
litellm/model_prices_and_context_window_backup.json
View file

@ -42258,14 +42258,14 @@
"supports_web_search": false
},
"openrouter/deepseek/deepseek-v4.1-flash": {
"cache_read_input_token_cost": 1e-08,
"input_cost_per_token": 3e-08,
"cache_read_input_token_cost": 6e-09,
"input_cost_per_token": 3e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
"max_tokens": 943718,
"mode": "chat",
"output_cost_per_token": 5e-07,
"output_cost_per_token": 1.2e-06,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -43604,19 +43604,19 @@
"supports_web_search": false
},
"openrouter/qwen/qwen3.5-35b-a3b": {
"input_cost_per_token": 1.625e-07,
"input_cost_per_token": 1.5e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 16384,
"max_tokens": 16384,
"max_output_tokens": 235929,
"max_tokens": 235929,
"mode": "chat",
"output_cost_per_token": 1.3e-06,
"output_cost_per_token": 1e-06,
"source": "https://openrouter.ai/api/v1/models",
"supports_function_calling": true,
"supports_reasoning": true,
"supports_tool_choice": true,
"supports_vision": true,
"cache_read_input_token_cost": 1.5625e-07,
"cache_read_input_token_cost": 5e-08,
"supports_audio_input": false,
"supports_pdf_input": false,
"supports_prompt_caching": true,
@ -43924,14 +43924,14 @@
},
"openrouter/z-ai/glm-5.1": {
"cache_creation_input_token_cost": 0.0,
"cache_read_input_token_cost": 1.7914e-07,
"input_cost_per_token": 9.646e-07,
"cache_read_input_token_cost": 2.6e-07,
"input_cost_per_token": 1.4e-06,
"litellm_provider": "openrouter",
"max_input_tokens": 204800,
"max_output_tokens": 131072,
"max_tokens": 131072,
"mode": "chat",
"output_cost_per_token": 3.0316e-06,
"output_cost_per_token": 4.4e-06,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -67419,13 +67419,13 @@
"supports_web_search": false
},
"openrouter/z-ai/glm-5.3": {
"input_cost_per_token": 2.219e-07,
"output_cost_per_token": 3.39e-06,
"cache_read_input_token_cost": 1.775e-07,
"input_cost_per_token": 1.4e-06,
"output_cost_per_token": 4.4e-06,
"cache_read_input_token_cost": 1.4e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
"max_tokens": 943718,
"max_output_tokens": 131072,
"max_tokens": 131072,
"mode": "chat",
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
@ -67556,8 +67556,8 @@
"supports_prompt_caching": true
},
"openrouter/deepseek/deepseek-v4-flash-0731": {
"cache_read_input_token_cost": 1.08e-08,
"input_cost_per_token": 1.08e-08,
"cache_read_input_token_cost": 5.1e-09,
"input_cost_per_token": 5.1e-09,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
@ -67609,6 +67609,7 @@
"input_cost_per_token": 9e-08,
"output_cost_per_token": 1.8e-07,
"cache_read_input_token_cost": 9e-09,
"deprecation_date": "2026-10-31",
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 131072,
@ -67626,6 +67627,7 @@
"supports_web_search": false
},
"openrouter/poolside/laguna-s-2.1:free": {
"deprecation_date": "2026-10-31",
"input_cost_per_token": 0.0,
"output_cost_per_token": 0.0,
"litellm_provider": "openrouter",
@ -67645,14 +67647,14 @@
"supports_web_search": false
},
"openrouter/moonshotai/kimi-k3": {
"cache_read_input_token_cost": 4.357e-07,
"input_cost_per_token": 4.357e-07,
"cache_read_input_token_cost": 2.7e-07,
"input_cost_per_token": 2.7e-06,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
"max_tokens": 943718,
"mode": "chat",
"output_cost_per_token": 1e-05,
"output_cost_per_token": 1.35e-05,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -67668,6 +67670,7 @@
"input_cost_per_token": 6e-08,
"output_cost_per_token": 1.2e-07,
"cache_read_input_token_cost": 3e-08,
"deprecation_date": "2026-10-31",
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 32768,
@ -67685,6 +67688,7 @@
"supports_web_search": false
},
"openrouter/poolside/laguna-xs-2.1:free": {
"deprecation_date": "2026-10-31",
"input_cost_per_token": 0.0,
"output_cost_per_token": 0.0,
"litellm_provider": "openrouter",
@ -67867,13 +67871,13 @@
"supports_web_search": false
},
"openrouter/nvidia/nemotron-3-ultra-550b-a55b": {
"input_cost_per_token": 6e-07,
"output_cost_per_token": 2.4e-06,
"cache_read_input_token_cost": 1.2e-07,
"input_cost_per_token": 5e-07,
"output_cost_per_token": 2.2e-06,
"cache_read_input_token_cost": 1e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 182520,
"max_tokens": 182520,
"max_output_tokens": 16384,
"max_tokens": 16384,
"mode": "chat",
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
@ -68131,14 +68135,14 @@
"supports_web_search": true
},
"openrouter/deepseek/deepseek-v4-flash": {
"cache_read_input_token_cost": 8.372e-09,
"input_cost_per_token": 4.186e-08,
"cache_read_input_token_cost": 5.6e-09,
"input_cost_per_token": 2.8e-08,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 131072,
"max_tokens": 131072,
"max_output_tokens": 384000,
"max_tokens": 384000,
"mode": "chat",
"output_cost_per_token": 8.372e-08,
"output_cost_per_token": 5.6e-08,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -68172,14 +68176,14 @@
"supports_web_search": false
},
"openrouter/google/gemma-4-26b-a4b-it": {
"cache_read_input_token_cost": 4.25e-08,
"input_cost_per_token": 7.65e-08,
"cache_read_input_token_cost": 3.75e-08,
"input_cost_per_token": 6.75e-08,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 235929,
"max_tokens": 235929,
"mode": "chat",
"output_cost_per_token": 2.55e-07,
"output_cost_per_token": 2.25e-07,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -68958,11 +68962,11 @@
"openrouter/deepseek/deepseek-v3.1-terminus": {
"cache_read_input_token_cost": 1.35e-07,
"deprecation_date": "2026-09-28",
"input_cost_per_token": 3e-07,
"input_cost_per_token": 2.7e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 163840,
"max_output_tokens": 65536,
"max_tokens": 65536,
"max_output_tokens": 147456,
"max_tokens": 147456,
"mode": "chat",
"output_cost_per_token": 1e-06,
"source": "https://openrouter.ai/api/v1/models",
@ -69006,12 +69010,13 @@
"supports_web_search": false
},
"openrouter/qwen/qwen3-next-80b-a3b-thinking": {
"deprecation_date": "2026-10-09",
"input_cost_per_token": 1.5e-07,
"output_cost_per_token": 1.2e-06,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 235929,
"max_tokens": 235929,
"max_output_tokens": 32768,
"max_tokens": 32768,
"mode": "chat",
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
@ -69187,13 +69192,13 @@
"supports_web_search": false
},
"openrouter/qwen/qwen3-30b-a3b-instruct-2507": {
"input_cost_per_token": 4.815e-08,
"input_cost_per_token": 1e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 32000,
"max_tokens": 32000,
"max_output_tokens": 235929,
"max_tokens": 235929,
"mode": "chat",
"output_cost_per_token": 1.9305e-07,
"output_cost_per_token": 3e-07,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -74319,14 +74324,14 @@
"supports_web_search": false
},
"openrouter/inclusionai/ling-3.0-flash-fin": {
"cache_read_input_token_cost": 1.2e-08,
"input_cost_per_token": 6e-08,
"cache_read_input_token_cost": 8.4e-09,
"input_cost_per_token": 4.2e-08,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 235929,
"max_tokens": 235929,
"max_output_tokens": 32768,
"max_tokens": 32768,
"mode": "chat",
"output_cost_per_token": 1.8e-07,
"output_cost_per_token": 1.232e-07,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -76406,12 +76411,12 @@
"supports_web_search": false
},
"openrouter/thinkingmachines/inkling": {
"cache_read_input_token_cost": 1.7e-07,
"input_cost_per_token": 1e-06,
"cache_read_input_token_cost": 1.6e-07,
"input_cost_per_token": 9.5e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 524288,
"max_output_tokens": 471859,
"max_tokens": 471859,
"max_output_tokens": 262144,
"max_tokens": 262144,
"mode": "chat",
"output_cost_per_token": 4.05e-06,
"source": "https://openrouter.ai/api/v1/models",
@ -79675,5 +79680,24 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true
},
"openrouter/inclusionai/ling-3.1-flash": {
"input_cost_per_token": 0.0,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 32768,
"max_tokens": 32768,
"mode": "chat",
"output_cost_per_token": 0.0,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
"supports_pdf_input": false,
"supports_prompt_caching": false,
"supports_reasoning": true,
"supports_response_schema": false,
"supports_tool_choice": true,
"supports_vision": false,
"supports_web_search": false
}
}

28
litellm/proxy/common_utils/registry_read_through.py
View file

@ -36,6 +36,7 @@ READ_THROUGH_MAX_RESYNCS_PER_WINDOW: Final = 20
class RegistryReadThrough:
__slots__ = (
"_is_loaded",
"_lock",
"_max_resyncs_per_window",
"_miss_ttl_seconds",
@ -49,11 +50,13 @@ class RegistryReadThrough:
def __init__(
self,
resync: Callable[[str], Awaitable[bool]],
is_loaded: Callable[[str], bool],
miss_ttl_seconds: float = READ_THROUGH_MISS_TTL_SECONDS,
max_resyncs_per_window: int = READ_THROUGH_MAX_RESYNCS_PER_WINDOW,
resync_window_seconds: float = READ_THROUGH_RESYNC_WINDOW_SECONDS,
) -> None:
self._resync = resync
self._is_loaded = is_loaded
self._miss_ttl_seconds = miss_ttl_seconds
self._max_resyncs_per_window = max_resyncs_per_window
self._resync_window_seconds = resync_window_seconds
@ -78,6 +81,8 @@ class RegistryReadThrough:
async with self._lock:
if self._recent_misses.get_cache(key) is not None:
return False
if self._is_loaded(key):
return True
if not self._consume_resync_budget():
verbose_proxy_logger.warning(
"registry read-through for %r skipped: resync budget of %s per %ss exhausted",
@ -190,9 +195,26 @@ async def _resync_agents(agent_id_or_name: str) -> bool:
return True
model_registry_read_through: Final = RegistryReadThrough(resync=_resync_model_deployments)
guardrail_registry_read_through: Final = RegistryReadThrough(resync=_resync_guardrails)
agent_registry_read_through: Final = RegistryReadThrough(resync=_resync_agents)
def _model_is_loaded(model_name_or_id: str) -> bool:
from litellm.proxy import proxy_server
router: Final = proxy_server.llm_router
if router is None:
return False
return model_name_or_id in router.model_names or router.has_model_id(model_name_or_id)
def _guardrail_is_loaded(guardrail_name: str) -> bool:
return _initialized_guardrail(guardrail_name) is not None
def _agent_is_loaded(agent_id_or_name: str) -> bool:
return _agent_from_registry(agent_id_or_name) is not None
model_registry_read_through: Final = RegistryReadThrough(resync=_resync_model_deployments, is_loaded=_model_is_loaded)
guardrail_registry_read_through: Final = RegistryReadThrough(resync=_resync_guardrails, is_loaded=_guardrail_is_loaded)
agent_registry_read_through: Final = RegistryReadThrough(resync=_resync_agents, is_loaded=_agent_is_loaded)
def _agent_from_registry(agent_id_or_name: str) -> "AgentResponse | None":

31
litellm/proxy/lens/worker.py
View file

@ -8,6 +8,7 @@ from types import MappingProxyType
from typing import Final
import httpx
from pydantic import BaseModel, ConfigDict, ValidationError
from .analysis import analyze_sample
from .models import Claim, Coverage, ExecutionContent, ModelRequest, ModelResult, Progress, Result, Sample
@ -15,6 +16,17 @@ from .models import Claim, Coverage, ExecutionContent, ModelRequest, ModelResult
logger: Final = logging.getLogger("litellm.lens.worker")
class ClaimedJobIdentity(BaseModel):
model_config = ConfigDict(frozen=True, extra="ignore")
id: str
class ClaimIdentity(BaseModel):
model_config = ConfigDict(frozen=True, extra="ignore")
lens_id: str
job: ClaimedJobIdentity
def failure_message(error: Exception) -> str:
if isinstance(error, (OSError, sqlite3.Error)):
return "Worker temporary storage failed. Increase its capacity or reduce analysis parallelism."
@ -74,9 +86,24 @@ class LensWorker:
async def run_once(self) -> bool:
response: Final = await self.client.post("/lens/worker/claim", params=MappingProxyType({"protocol_version": 2}))
response.raise_for_status()
if response.json() is None:
payload: Final = response.json()
if payload is None:
return False
claim: Final = Claim.model_validate(response.json())
try:
claim: Final = Claim.model_validate(payload)
except ValidationError:
identity: Final = ClaimIdentity.model_validate(payload)
failure: Final = await self.client.post(
f"/lens/worker/{identity.lens_id}/{identity.job.id}/result",
json=Result(
coverage=Coverage(),
error="The worker could not read this investigation. Update the worker to match the gateway, then retry.",
).model_dump(),
)
if failure.status_code != 409:
failure.raise_for_status()
logger.warning("Worker could not read a claimed investigation; reported a version compatibility failure")
return True
prefix: Final = f"/lens/worker/{claim.lens_id}/{claim.job.id}"
async def model(body: ModelRequest) -> ModelResult:

2
litellm/rust_bridge/_native.pyi
View file

@ -41,7 +41,7 @@ class NativeTraceStorage:
def ensure_schema(self) -> Future[None]: ...
def insert_rows(self, table: str, rows: Sequence[Mapping[str, object]]) -> Future[None]: ...
def query_sql(self, sql: str, scope: QueryScope, secret: str) -> Future[str]: ...
def query_help(self, scope: QueryScope, secret: str) -> Future[str]: ...
def query_help(self, scope: QueryScope, secret: str) -> Future[JsonValue]: ...
def query(self, query: ReadQueryName, parameters: Mapping[str, str | int | float | Sequence[str]]) -> Future[str]: ...
@final

35
litellm/rust_bridge/trace_query_responses.py
View file

@ -1,9 +1,12 @@
from collections.abc import Mapping
from typing import Final
from typing import Final, Literal
from pydantic import BaseModel, ConfigDict, JsonValue
_RESPONSE_CONFIG: Final = ConfigDict(frozen=True, extra="allow")
_HELP_CONFIG: Final = ConfigDict(frozen=True, extra="forbid")
TraceTableName = Literal["otel_traces", "agent_traces_by_key", "spend_logs"]
MetadataValueType = Literal["array", "boolean", "integer", "null", "number", "object", "string"]
class TraceQueryColumn(BaseModel):
@ -28,14 +31,14 @@ class TraceSQLResponse(BaseModel):
class TraceQueryTable(BaseModel):
model_config = ConfigDict(frozen=True)
name: str
model_config = _HELP_CONFIG
name: TraceTableName
columns: tuple[TraceQueryColumn, ...]
class TraceQueryNormalizedField(BaseModel):
model_config = ConfigDict(frozen=True)
table: str
model_config = _HELP_CONFIG
table: TraceTableName
name: str
column: str
type: str
@ -43,15 +46,15 @@ class TraceQueryNormalizedField(BaseModel):
class TraceQueryMetadataField(BaseModel):
model_config = ConfigDict(frozen=True)
model_config = _HELP_CONFIG
path: tuple[str | int, ...]
types: tuple[str, ...]
types: tuple[MetadataValueType, ...]
expression: str
class TraceQueryMetadata(BaseModel):
model_config = ConfigDict(frozen=True)
table: str
model_config = _HELP_CONFIG
table: TraceTableName
column: str
fields: tuple[TraceQueryMetadataField, ...]
sampled_rows: int
@ -63,15 +66,15 @@ class TraceQueryMetadata(BaseModel):
class TraceQueryAttributeField(BaseModel):
model_config = ConfigDict(frozen=True)
model_config = _HELP_CONFIG
key: str
type: str
type: Literal["String"]
expression: str
class TraceQueryAttributes(BaseModel):
model_config = ConfigDict(frozen=True)
table: str
model_config = _HELP_CONFIG
table: TraceTableName
column: str
fields: tuple[TraceQueryAttributeField, ...]
truncated: bool
@ -81,7 +84,7 @@ class TraceQueryAttributes(BaseModel):
class TraceQueryRelationship(BaseModel):
model_config = ConfigDict(frozen=True)
model_config = _HELP_CONFIG
left: str
right: str
additional_predicates: str
@ -89,13 +92,13 @@ class TraceQueryRelationship(BaseModel):
class TraceQueryExample(BaseModel):
model_config = ConfigDict(frozen=True)
model_config = _HELP_CONFIG
name: str
sql: str
class TraceQueryHelp(BaseModel):
model_config = ConfigDict(frozen=True)
model_config = _HELP_CONFIG
dialect: str
access: str
response: str

13
litellm/rust_bridge/traces.py
View file

@ -2,7 +2,7 @@ from collections.abc import Awaitable, Mapping, Sequence
from dataclasses import dataclass
from typing import Final, Literal, Protocol, TypedDict, TypeVar, runtime_checkable
from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, ValidationError
from pydantic import BaseModel, ConfigDict, Field, JsonValue, TypeAdapter, ValidationError
from typing_extensions import ReadOnly
from litellm.rust_bridge.loader import get_native_bridge
@ -111,7 +111,7 @@ class NativeStore(Protocol):
def query_sql(self, sql: str, scope: QueryScope, secret: str) -> Awaitable[str]: ...
def query_help(self, scope: QueryScope, secret: str) -> Awaitable[str]: ...
def query_help(self, scope: QueryScope, secret: str) -> Awaitable[JsonValue]: ...
def query(
self, name: ReadQueryName, parameters: Mapping[str, str | int | float | Sequence[str]]
@ -189,6 +189,13 @@ def _decode_query_response(adapter: TypeAdapter[_ResponseT], body: str) -> _Resp
raise RuntimeError("Native trace query returned an invalid response") from error
def _validate_query_response(adapter: TypeAdapter[_ResponseT], value: JsonValue) -> _ResponseT:
try:
return adapter.validate_python(value)
except ValidationError as error:
raise RuntimeError("Native trace query returned an invalid response") from error
class ClickHouseStorage:
def __init__(self, config: TraceStorageConfig) -> None:
native: Final = _native()
@ -216,7 +223,7 @@ class ClickHouseStorage:
async def query_help(self, scope: QueryScope, secret: str) -> TraceQueryHelp:
result: Final = await self._native.query_help(scope, secret)
return _decode_query_response(_HELP_RESPONSE, result)
return _validate_query_response(_HELP_RESPONSE, result)
async def lens_sample(self, parameters: LensSampleParams) -> tuple[ExecutionRow, ...]:
return await self.query(LENS_SAMPLE, parameters)

128
model_prices_and_context_window.json
View file

@ -42258,14 +42258,14 @@
"supports_web_search": false
},
"openrouter/deepseek/deepseek-v4.1-flash": {
"cache_read_input_token_cost": 1e-08,
"input_cost_per_token": 3e-08,
"cache_read_input_token_cost": 6e-09,
"input_cost_per_token": 3e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
"max_tokens": 943718,
"mode": "chat",
"output_cost_per_token": 5e-07,
"output_cost_per_token": 1.2e-06,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -43604,19 +43604,19 @@
"supports_web_search": false
},
"openrouter/qwen/qwen3.5-35b-a3b": {
"input_cost_per_token": 1.625e-07,
"input_cost_per_token": 1.5e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 16384,
"max_tokens": 16384,
"max_output_tokens": 235929,
"max_tokens": 235929,
"mode": "chat",
"output_cost_per_token": 1.3e-06,
"output_cost_per_token": 1e-06,
"source": "https://openrouter.ai/api/v1/models",
"supports_function_calling": true,
"supports_reasoning": true,
"supports_tool_choice": true,
"supports_vision": true,
"cache_read_input_token_cost": 1.5625e-07,
"cache_read_input_token_cost": 5e-08,
"supports_audio_input": false,
"supports_pdf_input": false,
"supports_prompt_caching": true,
@ -43924,14 +43924,14 @@
},
"openrouter/z-ai/glm-5.1": {
"cache_creation_input_token_cost": 0.0,
"cache_read_input_token_cost": 1.7914e-07,
"input_cost_per_token": 9.646e-07,
"cache_read_input_token_cost": 2.6e-07,
"input_cost_per_token": 1.4e-06,
"litellm_provider": "openrouter",
"max_input_tokens": 204800,
"max_output_tokens": 131072,
"max_tokens": 131072,
"mode": "chat",
"output_cost_per_token": 3.0316e-06,
"output_cost_per_token": 4.4e-06,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -67419,13 +67419,13 @@
"supports_web_search": false
},
"openrouter/z-ai/glm-5.3": {
"input_cost_per_token": 2.219e-07,
"output_cost_per_token": 3.39e-06,
"cache_read_input_token_cost": 1.775e-07,
"input_cost_per_token": 1.4e-06,
"output_cost_per_token": 4.4e-06,
"cache_read_input_token_cost": 1.4e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
"max_tokens": 943718,
"max_output_tokens": 131072,
"max_tokens": 131072,
"mode": "chat",
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
@ -67556,8 +67556,8 @@
"supports_prompt_caching": true
},
"openrouter/deepseek/deepseek-v4-flash-0731": {
"cache_read_input_token_cost": 1.08e-08,
"input_cost_per_token": 1.08e-08,
"cache_read_input_token_cost": 5.1e-09,
"input_cost_per_token": 5.1e-09,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
@ -67609,6 +67609,7 @@
"input_cost_per_token": 9e-08,
"output_cost_per_token": 1.8e-07,
"cache_read_input_token_cost": 9e-09,
"deprecation_date": "2026-10-31",
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 131072,
@ -67626,6 +67627,7 @@
"supports_web_search": false
},
"openrouter/poolside/laguna-s-2.1:free": {
"deprecation_date": "2026-10-31",
"input_cost_per_token": 0.0,
"output_cost_per_token": 0.0,
"litellm_provider": "openrouter",
@ -67645,14 +67647,14 @@
"supports_web_search": false
},
"openrouter/moonshotai/kimi-k3": {
"cache_read_input_token_cost": 4.357e-07,
"input_cost_per_token": 4.357e-07,
"cache_read_input_token_cost": 2.7e-07,
"input_cost_per_token": 2.7e-06,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 943718,
"max_tokens": 943718,
"mode": "chat",
"output_cost_per_token": 1e-05,
"output_cost_per_token": 1.35e-05,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -67668,6 +67670,7 @@
"input_cost_per_token": 6e-08,
"output_cost_per_token": 1.2e-07,
"cache_read_input_token_cost": 3e-08,
"deprecation_date": "2026-10-31",
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 32768,
@ -67685,6 +67688,7 @@
"supports_web_search": false
},
"openrouter/poolside/laguna-xs-2.1:free": {
"deprecation_date": "2026-10-31",
"input_cost_per_token": 0.0,
"output_cost_per_token": 0.0,
"litellm_provider": "openrouter",
@ -67867,13 +67871,13 @@
"supports_web_search": false
},
"openrouter/nvidia/nemotron-3-ultra-550b-a55b": {
"input_cost_per_token": 6e-07,
"output_cost_per_token": 2.4e-06,
"cache_read_input_token_cost": 1.2e-07,
"input_cost_per_token": 5e-07,
"output_cost_per_token": 2.2e-06,
"cache_read_input_token_cost": 1e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 182520,
"max_tokens": 182520,
"max_output_tokens": 16384,
"max_tokens": 16384,
"mode": "chat",
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
@ -68131,14 +68135,14 @@
"supports_web_search": true
},
"openrouter/deepseek/deepseek-v4-flash": {
"cache_read_input_token_cost": 8.372e-09,
"input_cost_per_token": 4.186e-08,
"cache_read_input_token_cost": 5.6e-09,
"input_cost_per_token": 2.8e-08,
"litellm_provider": "openrouter",
"max_input_tokens": 1048576,
"max_output_tokens": 131072,
"max_tokens": 131072,
"max_output_tokens": 384000,
"max_tokens": 384000,
"mode": "chat",
"output_cost_per_token": 8.372e-08,
"output_cost_per_token": 5.6e-08,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -68172,14 +68176,14 @@
"supports_web_search": false
},
"openrouter/google/gemma-4-26b-a4b-it": {
"cache_read_input_token_cost": 4.25e-08,
"input_cost_per_token": 7.65e-08,
"cache_read_input_token_cost": 3.75e-08,
"input_cost_per_token": 6.75e-08,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 235929,
"max_tokens": 235929,
"mode": "chat",
"output_cost_per_token": 2.55e-07,
"output_cost_per_token": 2.25e-07,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -68958,11 +68962,11 @@
"openrouter/deepseek/deepseek-v3.1-terminus": {
"cache_read_input_token_cost": 1.35e-07,
"deprecation_date": "2026-09-28",
"input_cost_per_token": 3e-07,
"input_cost_per_token": 2.7e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 163840,
"max_output_tokens": 65536,
"max_tokens": 65536,
"max_output_tokens": 147456,
"max_tokens": 147456,
"mode": "chat",
"output_cost_per_token": 1e-06,
"source": "https://openrouter.ai/api/v1/models",
@ -69006,12 +69010,13 @@
"supports_web_search": false
},
"openrouter/qwen/qwen3-next-80b-a3b-thinking": {
"deprecation_date": "2026-10-09",
"input_cost_per_token": 1.5e-07,
"output_cost_per_token": 1.2e-06,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 235929,
"max_tokens": 235929,
"max_output_tokens": 32768,
"max_tokens": 32768,
"mode": "chat",
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
@ -69187,13 +69192,13 @@
"supports_web_search": false
},
"openrouter/qwen/qwen3-30b-a3b-instruct-2507": {
"input_cost_per_token": 4.815e-08,
"input_cost_per_token": 1e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 32000,
"max_tokens": 32000,
"max_output_tokens": 235929,
"max_tokens": 235929,
"mode": "chat",
"output_cost_per_token": 1.9305e-07,
"output_cost_per_token": 3e-07,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -74319,14 +74324,14 @@
"supports_web_search": false
},
"openrouter/inclusionai/ling-3.0-flash-fin": {
"cache_read_input_token_cost": 1.2e-08,
"input_cost_per_token": 6e-08,
"cache_read_input_token_cost": 8.4e-09,
"input_cost_per_token": 4.2e-08,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 235929,
"max_tokens": 235929,
"max_output_tokens": 32768,
"max_tokens": 32768,
"mode": "chat",
"output_cost_per_token": 1.8e-07,
"output_cost_per_token": 1.232e-07,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
@ -76406,12 +76411,12 @@
"supports_web_search": false
},
"openrouter/thinkingmachines/inkling": {
"cache_read_input_token_cost": 1.7e-07,
"input_cost_per_token": 1e-06,
"cache_read_input_token_cost": 1.6e-07,
"input_cost_per_token": 9.5e-07,
"litellm_provider": "openrouter",
"max_input_tokens": 524288,
"max_output_tokens": 471859,
"max_tokens": 471859,
"max_output_tokens": 262144,
"max_tokens": 262144,
"mode": "chat",
"output_cost_per_token": 4.05e-06,
"source": "https://openrouter.ai/api/v1/models",
@ -79675,5 +79680,24 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true
},
"openrouter/inclusionai/ling-3.1-flash": {
"input_cost_per_token": 0.0,
"litellm_provider": "openrouter",
"max_input_tokens": 262144,
"max_output_tokens": 32768,
"max_tokens": 32768,
"mode": "chat",
"output_cost_per_token": 0.0,
"source": "https://openrouter.ai/api/v1/models",
"supports_audio_input": false,
"supports_function_calling": true,
"supports_pdf_input": false,
"supports_prompt_caching": false,
"supports_reasoning": true,
"supports_response_schema": false,
"supports_tool_choice": true,
"supports_vision": false,
"supports_web_search": false
}
}

69
tests/e2e/ui/tests/logs/logs.spec.ts
View file

@ -54,6 +54,75 @@ test.describe("Logs page", () => {
permissions: ["clipboard-read", "clipboard-write"],
});
test("log tables fill the available height and empty requests stay centered after resizing", async ({
page,
}) => {
await navigateToPage(page, Page.Logs);
await dismissFeedbackPopup(page);
await visibleTestId(page, "datatable-search").fill(
`missing-request-${uniqueSuffix()}`,
);
const emptyTitle = page.getByText("No matching requests", { exact: true });
await expect(emptyTitle).toBeVisible();
for (const viewport of [
{ width: 1440, height: 900 },
{ width: 1024, height: 720 },
]) {
await page.setViewportSize(viewport);
await expect
.poll(async () => {
const frame = await visibleTestId(
page,
"data-table-frame",
).boundingBox();
return frame
? Math.abs(viewport.height - frame.y - frame.height - 24)
: Infinity;
})
.toBeLessThanOrEqual(2);
await expect
.poll(async () => {
const body = await page
.locator("table")
.filter({ visible: true })
.first()
.locator("tbody")
.boundingBox();
const scroller = await visibleTestId(
page,
"data-table-scroller",
).boundingBox();
const message = await emptyTitle.locator("..").boundingBox();
if (!body || !message || !scroller) return Infinity;
return Math.max(
Math.abs(
message.x + message.width / 2 - scroller.x - scroller.width / 2,
),
Math.abs(message.y + message.height / 2 - body.y - body.height / 2),
);
})
.toBeLessThanOrEqual(4);
for (const tab of ["Deleted Keys", "Deleted Teams"]) {
await page.getByRole("tab", { name: tab, exact: true }).click();
await expect
.poll(async () => {
const frame = await visibleTestId(
page,
"data-table-frame",
).boundingBox();
return frame
? Math.abs(viewport.height - frame.y - frame.height - 24)
: Infinity;
})
.toBeLessThanOrEqual(2);
}
await page
.getByRole("tab", { name: "Request Logs", exact: true })
.click();
}
});
test("a chat sent from the Playground lands in Logs with its content", async ({ page, request }) => {
const prompt = `logs-playground-prompt-${uniqueSuffix()}`;
await openPlayground(page);

166
tests/unit/proxy/common_utils/test_registry_read_through.py
View file

@ -1,10 +1,17 @@
import asyncio
from typing import Final
from typing import TYPE_CHECKING, Final
import pytest
from litellm.proxy.common_utils.registry_read_through import RegistryReadThrough
if TYPE_CHECKING:
from litellm.proxy.agent_endpoints.agent_registry import AgentRegistry
def nothing_loaded(_key: str) -> bool:
return False
class ResyncSpy:
def __init__(self, found: bool = True, error: Exception | None = None) -> None:
@ -22,7 +29,7 @@ class ResyncSpy:
@pytest.mark.asyncio
async def test_attempt_returns_true_when_resync_finds_object():
spy: Final = ResyncSpy(found=True)
read_through: Final = RegistryReadThrough(resync=spy)
read_through: Final = RegistryReadThrough(resync=spy, is_loaded=nothing_loaded)
assert await read_through.attempt("new-model") is True
assert spy.calls == ["new-model"]
@ -31,7 +38,7 @@ async def test_attempt_returns_true_when_resync_finds_object():
@pytest.mark.asyncio
async def test_attempt_found_key_is_not_negative_cached():
spy: Final = ResyncSpy(found=True)
read_through: Final = RegistryReadThrough(resync=spy)
read_through: Final = RegistryReadThrough(resync=spy, is_loaded=nothing_loaded)
assert await read_through.attempt("new-model") is True
assert await read_through.attempt("new-model") is True
@ -41,7 +48,7 @@ async def test_attempt_found_key_is_not_negative_cached():
@pytest.mark.asyncio
async def test_missing_key_is_negative_cached_within_ttl():
spy: Final = ResyncSpy(found=False)
read_through: Final = RegistryReadThrough(resync=spy, miss_ttl_seconds=60.0)
read_through: Final = RegistryReadThrough(resync=spy, is_loaded=nothing_loaded, miss_ttl_seconds=60.0)
assert await read_through.attempt("ghost-model") is False
assert await read_through.attempt("ghost-model") is False
@ -51,7 +58,7 @@ async def test_missing_key_is_negative_cached_within_ttl():
@pytest.mark.asyncio
async def test_negative_cache_expires_and_resync_runs_again():
spy: Final = ResyncSpy(found=False)
read_through: Final = RegistryReadThrough(resync=spy, miss_ttl_seconds=0.05)
read_through: Final = RegistryReadThrough(resync=spy, is_loaded=nothing_loaded, miss_ttl_seconds=0.05)
assert await read_through.attempt("ghost-model") is False
await asyncio.sleep(0.1)
@ -62,7 +69,7 @@ async def test_negative_cache_expires_and_resync_runs_again():
@pytest.mark.asyncio
async def test_resync_exception_returns_false_without_negative_caching():
spy: Final = ResyncSpy(error=RuntimeError("db down"))
read_through: Final = RegistryReadThrough(resync=spy)
read_through: Final = RegistryReadThrough(resync=spy, is_loaded=nothing_loaded)
assert await read_through.attempt("new-model") is False
assert await read_through.attempt("new-model") is False
@ -77,7 +84,7 @@ async def test_concurrent_attempts_for_missing_key_resync_once():
return await super().__call__(key)
spy: Final = SlowResyncSpy(found=False)
read_through: Final = RegistryReadThrough(resync=spy, miss_ttl_seconds=60.0)
read_through: Final = RegistryReadThrough(resync=spy, is_loaded=nothing_loaded, miss_ttl_seconds=60.0)
results: Final = await asyncio.gather(*(read_through.attempt("ghost-model") for _ in range(5)))
assert results == [False] * 5
@ -87,7 +94,7 @@ async def test_concurrent_attempts_for_missing_key_resync_once():
@pytest.mark.asyncio
async def test_distinct_keys_do_not_share_negative_cache():
spy: Final = ResyncSpy(found=False)
read_through: Final = RegistryReadThrough(resync=spy, miss_ttl_seconds=60.0)
read_through: Final = RegistryReadThrough(resync=spy, is_loaded=nothing_loaded, miss_ttl_seconds=60.0)
assert await read_through.attempt("ghost-a") is False
assert await read_through.attempt("ghost-b") is False
@ -98,7 +105,11 @@ async def test_distinct_keys_do_not_share_negative_cache():
async def test_resync_budget_exhausted_blocks_resync_without_negative_caching():
spy: Final = ResyncSpy(found=False)
read_through: Final = RegistryReadThrough(
resync=spy, miss_ttl_seconds=60.0, max_resyncs_per_window=2, resync_window_seconds=60.0
resync=spy,
is_loaded=nothing_loaded,
miss_ttl_seconds=60.0,
max_resyncs_per_window=2,
resync_window_seconds=60.0,
)
assert await read_through.attempt("ghost-a") is False
@ -108,10 +119,48 @@ async def test_resync_budget_exhausted_blocks_resync_without_negative_caching():
assert read_through._recent_misses.get_cache("ghost-c") is None
@pytest.mark.asyncio
async def test_requests_queued_behind_a_successful_resync_spend_no_budget():
from unittest.mock import AsyncMock, call
entered: Final = asyncio.Event()
release: Final = asyncio.Event()
new_model_loaded: Final = asyncio.Event()
async def gated_load(key: str) -> bool:
entered.set()
await release.wait()
if key == "new-model":
new_model_loaded.set()
return True
def is_loaded(key: str) -> bool:
return key == "new-model" and new_model_loaded.is_set()
resync: Final = AsyncMock(side_effect=gated_load)
read_through: Final = RegistryReadThrough(
resync=resync,
is_loaded=is_loaded,
max_resyncs_per_window=2,
resync_window_seconds=60.0,
)
burst: Final = asyncio.gather(*(read_through.attempt("new-model") for _ in range(25)))
await entered.wait()
release.set()
assert await burst == [True] * 25
assert resync.await_args_list == [call("new-model")]
assert await read_through.attempt("other-model") is True
assert resync.await_args_list == [call("new-model"), call("other-model")]
@pytest.mark.asyncio
async def test_resync_budget_replenishes_after_window():
spy: Final = ResyncSpy(found=True)
read_through: Final = RegistryReadThrough(resync=spy, max_resyncs_per_window=1, resync_window_seconds=0.05)
read_through: Final = RegistryReadThrough(
resync=spy, is_loaded=nothing_loaded, max_resyncs_per_window=1, resync_window_seconds=0.05
)
assert await read_through.attempt("model-a") is True
assert await read_through.attempt("model-b") is False
@ -551,3 +600,100 @@ async def test_agent_read_through_hydrates_identity_binding(lookup, clean_agent_
assert agent.identity is not None
assert agent.identity.model_dump(include=set(binding)) == binding
assert clean_agent_registry.get_agent_by_id(agent_id="agent-id").identity == agent.identity
def test_model_is_loaded_matches_router_model_names_and_deployment_ids(monkeypatch: pytest.MonkeyPatch):
import litellm.proxy.proxy_server as proxy_server
from litellm import Router
from litellm.proxy.common_utils.registry_read_through import _model_is_loaded
router: Final = Router(
model_list=[
{
"model_name": "loaded-model",
"litellm_params": {"model": "openai/gpt-4o-mini", "api_key": "sk-test"},
"model_info": {"id": "loaded-deployment-id"},
}
]
)
monkeypatch.setattr(proxy_server, "llm_router", router)
assert _model_is_loaded("loaded-model") is True
assert _model_is_loaded("loaded-deployment-id") is True
assert _model_is_loaded("model-created-on-a-sibling") is False
monkeypatch.setattr(proxy_server, "llm_router", None)
assert _model_is_loaded("loaded-model") is False
@pytest.mark.asyncio
async def test_model_read_through_answers_a_loaded_model_without_reading_the_db(monkeypatch: pytest.MonkeyPatch):
from unittest.mock import AsyncMock, MagicMock
import litellm.proxy.proxy_server as proxy_server
from litellm import Router
from litellm.proxy.common_utils.registry_read_through import model_registry_read_through
prisma_client: Final = MagicMock()
prisma_client.db.litellm_proxymodeltable.find_many = AsyncMock(side_effect=AssertionError("db read"))
router: Final = Router(
model_list=[
{
"model_name": "wired-loaded-model",
"litellm_params": {"model": "openai/gpt-4o-mini", "api_key": "sk-test"},
}
]
)
monkeypatch.setattr(proxy_server, "prisma_client", prisma_client)
monkeypatch.setattr(proxy_server, "store_model_in_db", True)
monkeypatch.setattr(proxy_server, "llm_router", router)
assert await model_registry_read_through.attempt("wired-loaded-model") is True
prisma_client.db.litellm_proxymodeltable.find_many.assert_not_awaited()
@pytest.mark.asyncio
async def test_guardrail_read_through_answers_a_loaded_guardrail_without_reading_the_db(
monkeypatch: pytest.MonkeyPatch,
):
from unittest.mock import AsyncMock, MagicMock
import litellm.proxy.proxy_server as proxy_server
from litellm.proxy.common_utils.registry_read_through import guardrail_registry_read_through
from litellm.proxy.guardrails.guardrail_registry import IN_MEMORY_GUARDRAIL_HANDLER
from litellm.types.guardrails import Guardrail
guardrail_id: Final = "wired-loaded-guardrail-id"
guardrail_name: Final = "wired-loaded-guardrail"
prisma_client: Final = MagicMock()
prisma_client.db.litellm_guardrailstable.find_first = AsyncMock(side_effect=AssertionError("db read"))
monkeypatch.setattr(proxy_server, "prisma_client", prisma_client)
monkeypatch.setattr(proxy_server, "store_model_in_db", True)
IN_MEMORY_GUARDRAIL_HANDLER.sync_guardrail_from_db(
guardrail=Guardrail(**dict(FakeGuardrailRow(guardrail_id, guardrail_name)))
)
try:
assert await guardrail_registry_read_through.attempt(guardrail_name) is True
prisma_client.db.litellm_guardrailstable.find_first.assert_not_awaited()
finally:
IN_MEMORY_GUARDRAIL_HANDLER.delete_in_memory_guardrail(guardrail_id)
@pytest.mark.asyncio
async def test_agent_read_through_answers_a_loaded_agent_without_reading_the_db(
clean_agent_registry: "AgentRegistry", monkeypatch: pytest.MonkeyPatch
):
import litellm.proxy.proxy_server as proxy_server
from litellm.proxy.common_utils.registry_read_through import agent_registry_read_through
from litellm.types.agents import AgentResponse
monkeypatch.setattr(proxy_server, "store_model_in_db", False)
clean_agent_registry.register_agent(
agent_config=AgentResponse.model_validate(
FakeAgentRow("wired-loaded-agent-id", "wired-loaded-agent").model_dump()
)
)
assert await agent_registry_read_through.attempt("wired-loaded-agent-id") is True
assert await agent_registry_read_through.attempt("wired-loaded-agent") is True

4
tests/unit/proxy/conftest.py
View file

@ -411,6 +411,8 @@ def create_proxy_test_client(
def fresh_agent_read_through(monkeypatch):
from litellm.proxy.common_utils import registry_read_through
read_through = registry_read_through.RegistryReadThrough(resync=registry_read_through._resync_agents)
read_through = registry_read_through.RegistryReadThrough(
resync=registry_read_through._resync_agents, is_loaded=registry_read_through._agent_is_loaded
)
monkeypatch.setattr(registry_read_through, "agent_registry_read_through", read_through)
return read_through

40
tests/unit/proxy/lens/test_worker.py
View file

@ -3,6 +3,7 @@ from typing import Final
import httpx
import pytest
from pydantic import ValidationError
from litellm.proxy.lens.models import (
Claim,
@ -81,6 +82,45 @@ async def test_idle_worker_does_not_start_an_analysis() -> None:
assert await LensWorker(client).run_once() is False
@pytest.mark.asyncio
@pytest.mark.parametrize("result_status", (200, 409))
async def test_incompatible_claim_reports_failure_instead_of_leaving_the_investigation_running(
result_status: int,
) -> None:
claim: Final = Claim(lens_id="lens", job=queue_job(lens(), NOW, "job").jobs[0], findings=())
payload: Final = claim.model_dump(mode="json") | {
"job": claim.job.model_dump(mode="json") | {
"settings": claim.job.settings.model_dump() | {"future_setting": "private content"},
},
}
saved: Final = SimpleQueue[Result]()
def handle(request: httpx.Request) -> httpx.Response:
if request.url.path == "/lens/worker/claim":
return httpx.Response(200, json=payload)
assert request.url.path == "/lens/worker/lens/job/result"
saved.put(Result.model_validate_json(request.content))
return httpx.Response(result_status, json=True)
async with httpx.AsyncClient(base_url="https://proxy.test", transport=httpx.MockTransport(handle)) as client:
assert await LensWorker(client).run_once() is True
assert saved.get_nowait().error == (
"The worker could not read this investigation. Update the worker to match the gateway, then retry."
)
assert saved.empty()
@pytest.mark.asyncio
async def test_claim_without_an_identity_does_not_report_failure_for_another_investigation() -> None:
def handle(request: httpx.Request) -> httpx.Response:
assert request.url.path == "/lens/worker/claim"
return httpx.Response(200, json={"job": {"settings": {"future_setting": True}}})
async with httpx.AsyncClient(base_url="https://proxy.test", transport=httpx.MockTransport(handle)) as client:
with pytest.raises(ValidationError):
await LensWorker(client).run_once()
@pytest.mark.asyncio
@pytest.mark.parametrize("model_status", (200, 402, 503))
async def test_worker_reads_claimed_activity_and_reports_analysis_or_failure(model_status: int) -> None:

69
tests/unit/proxy/test_tracing_endpoints.py
View file

@ -2,8 +2,9 @@
Tests for the agent tracing endpoints (litellm/proxy/tracing_endpoints.py).
"""
from collections.abc import AsyncGenerator
from collections.abc import AsyncGenerator, Mapping
from contextlib import asynccontextmanager
from types import ModuleType
from typing import Final
from unittest.mock import AsyncMock, MagicMock
@ -15,9 +16,10 @@ from litellm.proxy import tracing_endpoints
from litellm.proxy._types import LitellmUserRoles, ProxyLifespanState, UserAPIKeyAuth
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
from litellm.proxy.tracing_runtime import manage_tracing, provide_storage
from litellm.rust_bridge import loader
from litellm.rust_bridge.trace_queries import SPAN_DETAIL, SpanDetailParams
from litellm.rust_bridge.trace_query_responses import TraceQueryHelp, TraceSQLResponse
from litellm.rust_bridge.traces import ClickHouseStorage
from litellm.rust_bridge.traces import AdminQueryScope, ClickHouseStorage, TraceStorageConfig
from litellm.tracing import TraceReceiver, TracingPayloadTooLargeError
from litellm.tracing.store import TraceStore
from litellm.tracing.types import TraceScope
@ -29,14 +31,14 @@ SQL_ENVELOPE: Final = {
"statistics": {"elapsed": 0.01, "rows_read": 1, "bytes_read": 8},
"rows_before_limit_at_least": 1,
}
QUERY_HELP: Final = {
QUERY_HELP: Final[Mapping[str, object]] = {
"dialect": "test SQL",
"access": "authenticated scope",
"response": "JSON envelope",
"tables": [{"name": "traces", "columns": [{"name": "value", "type": "String", "comment": "label"}]}],
"tables": [{"name": "otel_traces", "columns": [{"name": "value", "type": "String", "comment": "label"}]}],
"normalized_fields": [],
"metadata": {
"table": "traces",
"table": "spend_logs",
"column": "metadata",
"fields": [],
"sampled_rows": 0,
@ -668,3 +670,60 @@ def test_queries_require_a_proxy_secret(
receiver.store.storage.query_sql.assert_awaited_once_with(
"SELECT 1", {"kind": "logs", "user_id": "", "team_ids": (), "api_key_hash": "hashed-key"}, secret
)
class _NativeConfig:
def __init__(self, database: str, url: str, retention_days: int) -> None:
pass
class _NativeReturningHelp(ModuleType):
def __init__(self, help_payload: Mapping[str, object]) -> None:
super().__init__("native_traces")
class Storage:
def __init__(self, config: _NativeConfig) -> None:
pass
async def query_help(self, scope: AdminQueryScope, secret: str) -> Mapping[str, object]:
return help_payload
self.NativeTraceConfig: Final = _NativeConfig
self.NativeTraceStorage: Final = Storage
self.trace_decode_otlp: Final = list
self.trace_encode_error: Final = bytes
self.trace_normalized_field_definitions: Final = list
async def test_storage_validates_the_native_query_help_value(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(loader, "_cached_bridge", _NativeReturningHelp(QUERY_HELP))
storage: Final = ClickHouseStorage(TraceStorageConfig("http://clickhouse:8123"))
assert await storage.query_help({"kind": "admin"}, "secret") == TraceQueryHelp.model_validate(QUERY_HELP)
@pytest.mark.parametrize(
"drift",
(
{
"metadata": {
"table": "spend_logs",
"column": "metadata",
"fields": [{"path": ["a"], "types": ["boolen"], "expression": "a"}],
"sampled_rows": 1,
"invalid_json_rows": 0,
"truncated": False,
"sample_sql": "SELECT metadata FROM spend_logs",
"scope": "bounded sample",
}
},
{"tables": [{"name": "traces", "columns": [{"name": "value", "type": "String"}]}]},
{"unexpected": True},
),
)
async def test_storage_rejects_native_query_help_that_drifts_from_the_contract(
monkeypatch: pytest.MonkeyPatch, drift: Mapping[str, object]
) -> None:
monkeypatch.setattr(loader, "_cached_bridge", _NativeReturningHelp({**QUERY_HELP, **drift}))
storage: Final = ClickHouseStorage(TraceStorageConfig("http://clickhouse:8123"))
with pytest.raises(RuntimeError, match="invalid response"):
await storage.query_help({"kind": "admin"}, "secret")

BIN
ui/litellm-dashboard/public/assets/logos/google-adk.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
ui/litellm-dashboard/public/assets/logos/hermes.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.1 KiB

BIN
ui/litellm-dashboard/public/assets/logos/openclaw.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1,015 B

4
ui/litellm-dashboard/public/assets/logos/strands.svg Normal file
View file

@ -0,0 +1,4 @@
<svg width="290" height="463" viewBox="0 0 290 463" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M97.2902 52.7884C85.0674 49.1667 72.2234 56.1389 68.6017 68.3616C64.9801 80.5843 71.9524 93.4283 84.1749 97.0501L235.117 139.775C245.223 142.769 246.357 156.628 236.874 161.226L32.546 260.291C-14.9439 283.316 -9.16107 352.74 41.4835 367.591L189.551 411.009L190.125 411.169C202.183 414.376 214.665 407.396 218.196 395.355C221.784 383.122 214.774 370.296 202.541 366.709L54.4738 323.291C44.3447 320.321 43.1879 306.436 52.6857 301.831L257.014 202.766C304.432 179.776 298.758 110.483 248.233 95.512L97.2902 52.7884Z" fill="#989898"/>
<path d="M259.147 0.981812C271.389 -2.57498 284.197 4.46571 287.754 16.7074C291.311 28.9492 284.27 41.757 272.028 45.3138L71.1727 103.671C40.7142 112.521 37.1976 154.262 65.7459 168.083L241.343 253.093C307.872 285.302 299.794 382.546 228.862 403.336L30.4041 461.502C18.1707 465.088 5.34708 458.078 1.76153 445.844C-1.8239 433.611 5.18637 420.787 17.4197 417.202L215.878 359.035C246.277 350.125 249.739 308.449 221.226 294.645L45.6297 209.635C-20.9834 177.386 -12.7772 79.9893 58.2928 59.3402L259.147 0.981812Z" fill="#00FF77"/>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 1.1 KiB

81
ui/litellm-dashboard/src/app/(dashboard)/access-groups/_components/AccessGroupsPage.tsx
View file

@ -1,9 +1,10 @@
import { Page, PageContent } from "@/components/shared/Page";
import { AccessGroupResponse, useAccessGroups } from "@/app/(dashboard)/hooks/accessGroups/useAccessGroups";
import { useDeleteAccessGroup } from "@/app/(dashboard)/hooks/accessGroups/useDeleteAccessGroup";
import { Boxes, Plus, SearchIcon, X } from "lucide-react";
import { useMemo, useState } from "react";
import DeleteResourceModal from "@/components/common_components/DeleteResourceModal";
import { PageHeader } from "@/components/shared/PageHeader";
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { Button } from "@/components/ui/button";
import { InputGroup, InputGroupAddon, InputGroupButton, InputGroupInput } from "@/components/ui/input-group";
import { AccessGroupDetail } from "./AccessGroupsDetailsPage";
@ -59,49 +60,53 @@ export function AccessGroupsPage() {
}
return (
<div className="p-8">
<PageHeader
icon={<Boxes />}
title="Access Groups"
subtitle="Manage resource permissions for your organization"
primaryAction={
canModify ? (
<Page>
<PageHeader>
<PageHeaderTitle>
<Boxes />
Access Groups
</PageHeaderTitle>
<PageHeaderDescription>Manage resource permissions for your organization</PageHeaderDescription>
{canModify && (
<PageHeaderControls>
<Button onClick={() => setIsCreateModalVisible(true)}>
<Plus className="size-4" />
Create Access Group
</Button>
) : undefined
}
/>
</PageHeaderControls>
)}
</PageHeader>
<div className="mt-6 mb-3 flex items-center">
<InputGroup className="max-w-[400px]">
<InputGroupAddon>
<SearchIcon className="size-4 text-muted-foreground" />
</InputGroupAddon>
<InputGroupInput
placeholder="Search groups by name, ID, or description..."
value={searchText}
onChange={(e) => setSearchText(e.target.value)}
/>
{searchText && (
<InputGroupAddon align="inline-end">
<InputGroupButton size="icon-xs" aria-label="Clear search" onClick={() => setSearchText("")}>
<X />
</InputGroupButton>
<PageContent className="gap-3">
<div className="flex items-center">
<InputGroup className="max-w-[400px]">
<InputGroupAddon>
<SearchIcon className="size-4 text-muted-foreground" />
</InputGroupAddon>
)}
</InputGroup>
</div>
<InputGroupInput
placeholder="Search groups by name, ID, or description..."
value={searchText}
onChange={(e) => setSearchText(e.target.value)}
/>
{searchText && (
<InputGroupAddon align="inline-end">
<InputGroupButton size="icon-xs" aria-label="Clear search" onClick={() => setSearchText("")}>
<X />
</InputGroupButton>
</InputGroupAddon>
)}
</InputGroup>
</div>
<AccessGroupsTable
groups={filteredGroups}
isLoading={isLoading}
isFiltered={searchText.trim().length > 0}
canModify={canModify}
onGroupClick={setSelectedGroupId}
onDeleteClick={setGroupToDelete}
/>
<AccessGroupsTable
groups={filteredGroups}
isLoading={isLoading}
isFiltered={searchText.trim().length > 0}
canModify={canModify}
onGroupClick={setSelectedGroupId}
onDeleteClick={setGroupToDelete}
/>
</PageContent>
<AccessGroupCreateDialog open={isCreateModalVisible} onOpenChange={setIsCreateModalVisible} />
@ -126,6 +131,6 @@ export function AccessGroupsPage() {
}}
confirmLoading={deleteMutation.isPending}
/>
</div>
</Page>
);
}

5
ui/litellm-dashboard/src/app/(dashboard)/api-keys/ApiKeysDashboard.tsx
View file

@ -1,5 +1,6 @@
"use client";
import { Page } from "@/components/shared/Page";
import { teamListCall as v2TeamListCall } from "@/app/(dashboard)/hooks/teams/useTeams";
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
import { KeyResponse, Team } from "@/components/key_team_helpers/key_list";
@ -71,7 +72,7 @@ export default function ApiKeysDashboard() {
}, [accessToken, userID, userRole]);
return (
<main className="flex h-full flex-col p-8">
<Page className="h-full">
<VirtualKeysTable
headerActions={
isViewOnly ? undefined : (
@ -86,6 +87,6 @@ export default function ApiKeysDashboard() {
)
}
/>
</main>
</Page>
);
}

61
ui/litellm-dashboard/src/app/(dashboard)/budgets/_components/budget_panel.tsx
View file

@ -3,13 +3,15 @@
*
*/
import { Page, PageTabs, PageTabsList, PageTabsTrigger } from "@/components/shared/Page";
import { Plus, Wallet } from "lucide-react";
import React, { useCallback, useState } from "react";
import { Prism as SyntaxHighlighter } from "react-syntax-highlighter";
import { prism } from "react-syntax-highlighter/dist/esm/styles/prism";
import { useSyntaxTheme } from "@/hooks/useSyntaxTheme";
import { PageHeader } from "@/components/shared/PageHeader";
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { ToolbarSeparator } from "@/components/shared/ToolbarSeparator";
import { Button } from "@/components/ui/button";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import DeleteResourceModal from "@/components/common_components/DeleteResourceModal";
@ -78,35 +80,30 @@ const BudgetPanel: React.FC<BudgetSettingsPageProps> = ({ accessToken }) => {
};
return (
<main className="flex h-full flex-col p-8">
<Tabs defaultValue="budgets" className="min-h-0 flex-1 gap-6">
<PageHeader
icon={<Wallet />}
title="Budgets"
subtitle="Spend, TPM and RPM limits you can assign to customers."
primaryAction={
canModify ? (
<Button onClick={() => setIsCreateModelVisible(true)}>
<Plus className="size-4" />
Create Budget
</Button>
) : undefined
}
tabs={({ leadingControls }) => (
<TabsList
variant="line"
className="gap-0 p-0 [&>[data-slot=tabs-trigger]+[data-slot=tabs-trigger]]:ml-[22px]"
>
{leadingControls}
<TabsTrigger value="budgets" className="flex-none px-0 py-[7px] data-active:font-semibold">
Budgets
</TabsTrigger>
<TabsTrigger value="examples" className="flex-none px-0 py-[7px] data-active:font-semibold">
Examples
</TabsTrigger>
</TabsList>
)}
/>
<Page className="h-full">
<PageTabs defaultValue="budgets">
<PageHeader>
<PageHeaderTitle>
<Wallet />
Budgets
</PageHeaderTitle>
<PageHeaderDescription>Spend, TPM and RPM limits you can assign to customers.</PageHeaderDescription>
<PageHeaderControls>
<PageTabsList>
{canModify && (
<>
<Button onClick={() => setIsCreateModelVisible(true)}>
<Plus className="size-4" />
Create Budget
</Button>
<ToolbarSeparator className="mx-0 h-6" />
</>
)}
<PageTabsTrigger value="budgets">Budgets</PageTabsTrigger>
<PageTabsTrigger value="examples">Examples</PageTabsTrigger>
</PageTabsList>
</PageHeaderControls>
</PageHeader>
<TabsContent value="budgets" className="flex min-h-0 flex-1 flex-col" keepMounted>
<div className="flex min-h-0 flex-1 flex-col">
<BudgetModal isModalVisible={isCreateModelVisible} setIsModalVisible={setIsCreateModelVisible} />
@ -174,8 +171,8 @@ const BudgetPanel: React.FC<BudgetSettingsPageProps> = ({ accessToken }) => {
</Tabs>
</div>
</TabsContent>
</Tabs>
</main>
</PageTabs>
</Page>
);
};

56
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/CostOptimizationView.tsx
View file

@ -1,12 +1,13 @@
"use client";
import { Page, PageTabs, PageTabsList, PageTabsTrigger } from "@/components/shared/Page";
import React from "react";
import { Info, PiggyBank } from "lucide-react";
import useCan from "@/app/(dashboard)/hooks/useCan";
import { Alert, AlertDescription } from "@/components/shared/Alert";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { PageHeader } from "@/components/shared/PageHeader";
import { TabsContent } from "@/components/ui/tabs";
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import UsageTab from "./UsageTab";
import PromptCompressionTab from "./PromptCompressionTab";
import PromptCachingTab from "./PromptCachingTab";
@ -33,37 +34,30 @@ const CostOptimizationView: React.FC<CostOptimizationViewProps> = ({ accessToken
};
return (
<main className="w-full p-8">
<Tabs defaultValue="usage" onValueChange={handleTabChange} className="gap-6">
<PageHeader
icon={<PiggyBank />}
title="Cost Optimization"
subtitle="Track and configure the mechanisms that save you money: prompt compression and prompt caching. Auto routers live under Models + Endpoints, on the Auto-Routers tab"
tabs={({ leadingControls }) => (
<TabsList
variant="line"
className="gap-0 p-0 [&>[data-slot=tabs-trigger]+[data-slot=tabs-trigger]]:ml-[22px]"
>
{leadingControls}
<TabsTrigger value="usage" className="flex-none px-0 py-[7px] data-active:font-semibold">
Overall
</TabsTrigger>
<Page>
<PageTabs defaultValue="usage" onValueChange={handleTabChange}>
<PageHeader>
<PageHeaderTitle>
<PiggyBank />
Cost Optimization
</PageHeaderTitle>
<PageHeaderDescription>
Track and configure the mechanisms that save you money: prompt compression and prompt caching. Auto routers
live under Models + Endpoints, on the Auto-Routers tab
</PageHeaderDescription>
<PageHeaderControls>
<PageTabsList>
<PageTabsTrigger value="usage">Overall</PageTabsTrigger>
{canViewProxyWideCostData && (
<>
<TabsTrigger value="compression" className="flex-none px-0 py-[7px] data-active:font-semibold">
Prompt Compression
</TabsTrigger>
<TabsTrigger value="caching" className="flex-none px-0 py-[7px] data-active:font-semibold">
Prompt Caching
</TabsTrigger>
<TabsTrigger value="autorouter-usage" className="flex-none px-0 py-[7px] data-active:font-semibold">
Auto-Router
</TabsTrigger>
<PageTabsTrigger value="compression">Prompt Compression</PageTabsTrigger>
<PageTabsTrigger value="caching">Prompt Caching</PageTabsTrigger>
<PageTabsTrigger value="autorouter-usage">Auto-Router</PageTabsTrigger>
</>
)}
</TabsList>
)}
/>
</PageTabsList>
</PageHeaderControls>
</PageHeader>
<div
role="alert"
@ -108,8 +102,8 @@ const CostOptimizationView: React.FC<CostOptimizationViewProps> = ({ accessToken
</TabsContent>
</>
)}
</Tabs>
</main>
</PageTabs>
</Page>
);
};

5
ui/litellm-dashboard/src/app/(dashboard)/guardrails-monitor/_components/GuardrailsMonitorView.tsx
View file

@ -1,3 +1,4 @@
import { Page } from "@/components/shared/Page";
import type { DateRangePickerValue } from "@/components/shared/date_picker_types";
import { parseAsString, useQueryState } from "nuqs";
import React, { useCallback, useMemo, useState } from "react";
@ -48,7 +49,7 @@ export default function GuardrailsMonitorView({ accessToken = null }: Guardrails
);
return (
<main className="w-full min-w-0 flex-1 p-8">
<Page>
{!selectedGuardrailId ? (
<GuardrailsOverview
accessToken={accessToken}
@ -69,6 +70,6 @@ export default function GuardrailsMonitorView({ accessToken = null }: Guardrails
/>
</>
)}
</main>
</Page>
);
}

30
ui/litellm-dashboard/src/app/(dashboard)/guardrails-monitor/_components/GuardrailsOverview.tsx
View file

@ -18,7 +18,7 @@ import {
type UsageUnits,
} from "@/components/GuardrailsMonitor/usageUnits";
import { Button } from "@/components/ui/button";
import { PageHeader } from "@/components/shared/PageHeader";
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { UiLoadingSpinner } from "@/components/ui/ui-loading-spinner";
import { EvaluationSettingsModal } from "./EvaluationSettingsModal";
import { MetricCard } from "@/components/GuardrailsMonitor/MetricCard";
@ -282,20 +282,20 @@ export function GuardrailsOverview({
return (
<div>
<PageHeader
icon={<HeartPulse />}
title="Guardrails Monitor"
subtitle="Monitor guardrail performance across all requests"
utilities={
<>
{dateRangeControl}
<Button variant="outline" title="Coming soon">
<Download className="size-4" />
Export Data
</Button>
</>
}
/>
<PageHeader>
<PageHeaderTitle>
<HeartPulse />
Guardrails Monitor
</PageHeaderTitle>
<PageHeaderDescription>Monitor guardrail performance across all requests</PageHeaderDescription>
<PageHeaderControls className="justify-end">
{dateRangeControl}
<Button variant="outline" title="Coming soon">
<Download className="size-4" />
Export Data
</Button>
</PageHeaderControls>
</PageHeader>
<div className="mt-6 mb-6 grid grid-cols-[repeat(auto-fit,minmax(7rem,1fr))] gap-4">
<MetricCard label="Total Evaluations" value={metrics.totalRequests.toLocaleString()} />

3
ui/litellm-dashboard/src/app/(dashboard)/hooks/models/useModelCostMap.ts
View file

@ -4,8 +4,9 @@ import { createQueryKeys } from "../common/queryKeysFactory";
const modelCostMapKeys = createQueryKeys("modelCostMap");
export const useModelCostMap = () => {
export const useModelCostMap = (enabled = true) => {
return useQuery<Record<string, any>>({
enabled,
queryKey: modelCostMapKeys.list({}),
queryFn: async () => await modelCostMap(),
staleTime: 60 * 1000, // 1 minute

2
ui/litellm-dashboard/src/app/(dashboard)/layout.tsx
View file

@ -149,7 +149,7 @@ function DashboardShell({ children }: { children: React.ReactNode }) {
<LicenseExpiryBanner accessToken={accessToken} />
<UserBanner accessToken={accessToken} />
<UpgradeBanner accessToken={accessToken} />
<main className="min-w-0 flex-1 overflow-y-auto">{children}</main>
<main className="min-h-0 min-w-0 flex-1 overflow-y-auto">{children}</main>
{!isPlayground && <LiteAdmin />}
</div>
</div>

58
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/LensOverview.tsx
View file

@ -67,64 +67,6 @@ export function InvestigationList({
);
}
export function InvestigationExample({ onClose }: { onClose: () => void }) {
return (
<Dialog
open
onOpenChange={(open) => {
if (!open) onClose();
}}
>
<DialogContent className="max-h-[90dvh] gap-5 overflow-y-auto sm:max-w-xl">
<DialogHeader className="gap-1.5 pr-6">
<DialogTitle className="text-base">Example investigation</DialogTitle>
</DialogHeader>
<section className="border-t pt-5" aria-labelledby="example-finding">
<div className="flex flex-wrap items-center justify-between gap-2 text-xs">
<span className="flex items-center gap-1.5 font-medium text-amber-700 dark:text-amber-400">
<span className="size-1.5 rounded-full bg-current" aria-hidden="true" />
Needs attention
</span>
<span className="text-muted-foreground">3 of 20 conversations</span>
</div>
<h3 id="example-finding" className="mt-2 text-lg font-semibold leading-6 tracking-tight">
Failed lookups leave customers without answers
</h3>
<DialogDescription className="mt-2 leading-5">
The agent retries the same failed order lookup, then ends the conversation without an answer or a handoff.
</DialogDescription>
</section>
<div className="overflow-hidden rounded-lg border">
<div className="space-y-1.5 bg-muted/30 px-4 py-3">
<p className="text-xs text-muted-foreground">After three failed lookups, the agent replies:</p>
<blockquote className="text-sm leading-5">“I will check that for you.”</blockquote>
</div>
<details className="group border-t">
<summary className="flex cursor-pointer list-none items-center justify-between gap-2 px-4 py-2.5 text-sm font-medium hover:bg-muted/30 focus-visible:outline-2 focus-visible:-outline-offset-2 focus-visible:outline-ring [&::-webkit-details-marker]:hidden">
See the trace <ChevronRight className="size-3.5 transition-transform group-open:rotate-90" />
</summary>
<ol className="space-y-3 border-t px-4 py-3 text-sm leading-5 text-muted-foreground">
<li>
<span className="font-medium text-foreground">Customer</span> · Where is my order?
</li>
<li>
<span className="font-medium text-foreground">Order lookup</span> · Service unavailable
</li>
<li>
<span className="font-medium text-foreground">Two retries</span> · Same error, no new information
</li>
<li>
<span className="font-medium text-foreground">Agent</span> · I will check that for you. Conversation
ends.
</li>
</ol>
</details>
</div>
</DialogContent>
</Dialog>
);
}
export function MonitoringSetup({
settings,
ready,

155
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/LensProgress.tsx
View file

@ -1,11 +1,31 @@
"use client";
import { useEffect, useState } from "react";
import { Check, Loader2 } from "lucide-react";
import { Button } from "@/components/ui/button";
import { analysisElapsed, analysisProgress, nextCheckStatus, type Lens, type Job } from "./lensData";
import {
analysisElapsed,
analysisFraction,
analysisPace,
analysisProgress,
analysisStages,
durationText,
nextCheckStatus,
remainingLabel,
stageDurations,
type Lens,
type Job,
type ProgressSample,
} from "./lensData";
const steps = ["Review runs", "Find patterns", "Check evidence"];
const steps = ["review runs", "find patterns", "check evidence"];
const markers = { done: "✓", active: "▸", todo: "·" };
const rowText = { done: "text-foreground", active: "font-medium text-foreground", todo: "text-muted-foreground" };
const blocks = 32;
function stageState(index: number, current: number): keyof typeof markers {
if (index < current) return "done";
return index === current ? "active" : "todo";
}
export function LensProgress({ job, onCancel }: { job: Job; onCancel?: () => void }) {
const [now, setNow] = useState(Date.now);
@ -14,57 +34,98 @@ export function LensProgress({ job, onCancel }: { job: Job; onCancel?: () => voi
return () => window.clearInterval(timer);
}, []);
const progress = analysisProgress(job);
const percent = progress.total ? Math.min(100, (progress.done / progress.total) * 100) : undefined;
const fraction = analysisFraction(progress);
const [samples, setSamples] = useState<ProgressSample[]>([]);
const latest = samples.at(-1);
if (!latest || latest.step !== progress.step || latest.done !== progress.done) {
setSamples([...samples, { at: now, step: progress.step, done: progress.done, fraction }].slice(-120));
}
const pace = analysisPace(samples, now);
const percent = Math.round(fraction * 100);
const queued = progress.step < 0;
const counts = analysisStages(job);
const durations = stageDurations(samples, job.created_at, now);
const filled = Math.round(fraction * blocks);
const stats = [
["eta", remainingLabel(pace.secondsLeft)],
["rate", pace.perMinute === null ? "–" : `${Math.round(pace.perMinute)}/min`],
["elapsed", analysisElapsed(job.created_at, now)],
];
return (
<section aria-label="Analysis progress" className="space-y-4 rounded-xl border bg-muted/30 p-4">
<div className="flex flex-wrap items-center justify-between gap-2">
<div className="flex items-center gap-2 text-sm font-medium" role="status">
<Loader2 aria-hidden="true" className="size-4 motion-safe:animate-spin text-muted-foreground" />
{progress.title}
</div>
<span className="text-xs tabular-nums text-muted-foreground">
{analysisElapsed(job.created_at, now)} elapsed
<section aria-label="Analysis progress" className="space-y-3 rounded-md border bg-muted/40 px-4 py-3 text-sm">
<div className="flex items-center justify-between gap-2">
<span className="truncate" role="status">
<span className="font-medium">{progress.title}</span>
{!queued && <span className="text-muted-foreground"> · {progress.detail}</span>}
</span>
</div>
<ol aria-label="Analysis stages" className="grid grid-cols-3 gap-2">
{steps.map((label, index) => (
<li key={label} aria-current={index === progress.step ? "step" : undefined} className="space-y-2">
<div className={`h-1 rounded-full ${index <= progress.step ? "bg-foreground" : "bg-border"}`} />
<span
className={`flex items-center gap-1 text-xs ${index === progress.step ? "font-medium" : "text-muted-foreground"}`}
>
{index < progress.step && <Check aria-label="Complete" className="size-3 shrink-0" />}
{label}
</span>
</li>
))}
</ol>
<div className="space-y-2">
<p className="text-xs text-muted-foreground">{progress.detail}</p>
<div
role="progressbar"
aria-label={progress.title}
aria-valuemin={0}
aria-valuemax={progress.total || undefined}
aria-valuenow={progress.total ? Math.min(progress.done, progress.total) : undefined}
aria-valuetext={progress.detail}
className="h-1.5 overflow-hidden rounded-full bg-muted"
>
<div
className={`h-full rounded-full bg-foreground/70 transition-[width] duration-500 ${percent === undefined ? "motion-safe:animate-pulse" : ""}`}
style={{ width: percent === undefined ? "100%" : `${percent}%` }}
/>
</div>
</div>
<div className="flex flex-wrap items-center justify-between gap-2 text-xs text-muted-foreground">
{job.status === "running" && <span>You can leave this page while the investigation runs.</span>}
{onCancel && (
<Button variant="ghost" size="sm" onClick={onCancel}>
Cancel analysis
<Button variant="ghost" size="sm" className="h-6 px-2 text-xs" onClick={onCancel}>
Cancel
</Button>
)}
</div>
<div className="max-w-2xl space-y-1.5 pl-3">
<div className="flex items-center gap-2">
<span aria-hidden="true" className="text-muted-foreground">
[
</span>
<div
role="progressbar"
aria-label="Investigation progress"
aria-valuemin={0}
aria-valuemax={100}
aria-valuenow={percent}
aria-valuetext={queued ? progress.title : `${progress.title}: ${progress.detail}`}
className="flex h-3.5 min-w-0 flex-1 gap-px"
>
{Array.from({ length: blocks }, (_, index) => (
<span
key={index}
className={`flex-1 ${index < filled ? "bg-foreground" : "bg-foreground/10"} ${
queued && index < blocks / 3 ? "motion-safe:animate-pulse bg-foreground/30" : ""
}`}
/>
))}
</div>
<span aria-hidden="true" className="text-muted-foreground">
]
</span>
<span className="w-10 text-right tabular-nums">{percent}%</span>
</div>
<ol aria-label="Analysis stages" className="space-y-0.5">
{steps.map((label, index) => {
const state = stageState(index, progress.step);
const { done, total } = counts[index];
const seconds = durations[index];
return (
<li
key={label}
aria-current={state === "active" ? "step" : undefined}
className={`grid grid-cols-[1rem_minmax(0,9rem)_6rem_auto] items-baseline gap-2 tabular-nums ${rowText[state]}`}
>
<span aria-hidden="true">{markers[state]}</span>
<span className="truncate">{label}</span>
<span className="text-right text-muted-foreground">
{state === "todo" || !total ? "–" : `${Math.min(done, total)}/${total}`}
</span>
<span className="text-muted-foreground">{seconds === null ? "" : durationText(seconds)}</span>
</li>
);
})}
</ol>
<div className="flex flex-wrap gap-x-5 gap-y-1 pt-1 tabular-nums">
{queued ? (
<span className="text-muted-foreground">{progress.detail}</span>
) : (
stats.map(([key, value]) => (
<span key={key}>
<span className="text-muted-foreground">{key}</span> {value}
</span>
))
)}
</div>
</div>
</section>
);
}

35
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/LensView.integration.test.tsx
View file

@ -231,39 +231,18 @@ it("runs saved settings immediately without opening setup", async () => {
expect(screen.queryByRole("dialog")).not.toBeInTheDocument();
});
it("lets a new user inspect example evidence and return to setup without starting an investigation", async () => {
it("offers the interactive demo without starting an investigation", async () => {
window.history.replaceState({}, "", "/lens/");
testQueryClient.clear();
vi.mocked(apiClient.get).mockImplementation(async (path) => {
if (path === "/lens") return { lenses: [], workers: [], tracing_enabled: false };
return { traces: false, requests: false };
});
const onDemo = vi.fn();
const user = userEvent.setup();
renderWithProviders(<LensView accessToken="test" />);
await user.click(await screen.findByRole("button", { name: "View an example" }));
const example = within(screen.getByRole("dialog", { name: "Example investigation" }));
expect(example.getByRole("heading", { name: "Failed lookups leave customers without answers" })).toBeVisible();
const evidence = example.getByText(/Where is my order\?/);
expect(evidence).not.toBeVisible();
await user.click(example.getByText("See the trace"));
expect(evidence).toBeVisible();
expect(example.getByText(/Service unavailable/)).toBeVisible();
await user.click(example.getByText("See the trace"));
expect(evidence).not.toBeVisible();
await user.click(example.getByRole("button", { name: "Close" }));
await waitFor(() => expect(screen.queryByRole("dialog")).not.toBeInTheDocument());
expect(screen.getByRole("link", { name: "Set up traces" })).toBeVisible();
expect(screen.getByRole("button", { name: "Connect worker" })).toBeDisabled();
expect(screen.getByRole("button", { name: "New investigation" })).toBeDisabled();
await user.click(screen.getByRole("button", { name: "View an example" }));
expect(screen.getByRole("dialog", { name: "Example investigation" })).toBeVisible();
expect(screen.getByText(/Where is my order\?/)).not.toBeVisible();
await user.keyboard("{Escape}");
await waitFor(() => expect(screen.queryByRole("dialog")).not.toBeInTheDocument());
expect(screen.getByRole("button", { name: "View an example" })).toBeVisible();
renderWithProviders(<LensView accessToken="test" onDemo={onDemo} />);
await user.click(await screen.findByRole("button", { name: "Preview sample" }));
expect(onDemo).toHaveBeenCalledOnce();
expect(apiClient.post).not.toHaveBeenCalled();
});
@ -276,13 +255,14 @@ it("guides a first-time administrator into worker connection and lens setup", as
return { traces: true, requests: false, data: [] };
});
const user = userEvent.setup();
renderWithProviders(<LensView accessToken="test" />);
renderWithProviders(<LensView accessToken="test" onDemo={vi.fn()} />);
const guide = within(await screen.findByRole("region", { name: "Find what needs attention" }));
expect(apiClient.get).toHaveBeenCalledWith("/lens/activity/available", { accessToken: "test" });
expect(await guide.findByRole("link", { name: "View traces" })).toHaveAttribute(
"href",
expect.stringMatching(/^\/ui\/lens\/?\?tab=traces$/),
);
expect(await guide.findByRole("button", { name: "Preview sample" })).toBeVisible();
await user.click(guide.getByRole("button", { name: "Connect worker" }));
const connection = within(await screen.findByRole("dialog", { name: "Connect a worker" }));
expect(connection.getByRole("button", { name: "Get install command" })).toBeVisible();
@ -305,6 +285,7 @@ it("guides a first-time administrator into worker connection and lens setup", as
});
});
await waitFor(() => expect(guide.getByRole("button", { name: "New investigation" })).toBeEnabled());
expect(guide.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
await user.click(guide.getByRole("button", { name: "New investigation" }));
expect(await screen.findByRole("dialog", { name: "Which activity should we investigate?" })).toBeVisible();
});

37
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/LensView.tsx
View file

@ -1,4 +1,5 @@
"use client";
import { useLensDemo } from "@/components/lens/LensDemoContext";
import type { components } from "@/lib/http/schema";
import { useEffect, useState } from "react";
@ -28,7 +29,7 @@ import { Tabs, TabsList, TabsTrigger, TabsContent } from "@/components/ui/tabs";
import { Sheet, SheetContent, SheetHeader, SheetTitle, SheetDescription } from "@/components/ui/sheet";
import { Popover, PopoverContent, PopoverTitle, PopoverTrigger } from "@/components/ui/popover";
import { LensFinding } from "./LensFinding";
import { apiClient } from "@/components/networking";
import { apiClient as liveApiClient } from "@/components/networking";
import { TracePanel } from "./TracePanel";
import { LensSetup } from "./LensSetup";
import { LensRuns } from "./LensRuns";
@ -37,13 +38,7 @@ import { WorkerSetup } from "./WorkerSetup";
import { useAnalysisKeyInfo } from "./AnalysisKeyDetails";
import { uiHref } from "@/utils/uiHref";
import { ApiError } from "@/lib/http/client";
import {
InvestigationList,
InvestigationExample,
MonitoringSetup,
InvestigationSummary,
InvestigationFailure,
} from "./LensOverview";
import { InvestigationList, MonitoringSetup, InvestigationSummary, InvestigationFailure } from "./LensOverview";
import { LensWelcome } from "./LensWelcome";
import {
workerConnected,
@ -70,10 +65,19 @@ function emptyFindingTitle(active: boolean, scanned: boolean, status?: string) {
return scanned ? "No matching findings" : "Ready for the first analysis";
}
export function LensView({ accessToken, readOnly = false }: { accessToken: string; readOnly?: boolean }) {
export function LensView({
accessToken,
readOnly = false,
onDemo,
}: {
accessToken: string;
readOnly?: boolean;
onDemo?: () => void;
}) {
const demo = useLensDemo();
const apiClient = demo?.client ?? liveApiClient;
const client = useQueryClient();
const [workerSetup, setWorkerSetup] = useState(false);
const [showExample, setShowExample] = useState(false);
const [monitoring, setMonitoring] = useState(false);
const [now, setNow] = useState(Date.now);
useEffect(() => {
@ -85,6 +89,7 @@ export function LensView({ accessToken, readOnly = false }: { accessToken: strin
queryKey: key,
queryFn: () => apiClient.get<LensList>("/lens", { accessToken }),
refetchInterval: (current) => {
if (demo) return false;
const running = current.state.data?.lenses.some((item) =>
item.jobs.some((job) => ["queued", "running"].includes(job.status)),
);
@ -100,7 +105,10 @@ export function LensView({ accessToken, readOnly = false }: { accessToken: strin
queryFn: () =>
apiClient.get<{ data: import("./lensData").AnalysisModelInfo[] }>("/model_group/info", { accessToken }),
});
const [selected, setSelected] = useQueryState("lens", parseAsString.withOptions({ history: "push" }));
const [liveSelected, setLiveSelected] = useQueryState("lens", parseAsString.withOptions({ history: "push" }));
const [demoSelected, setDemoSelected] = useState<string | null>(null);
const selected = demo ? demoSelected : liveSelected;
const setSelected = demo ? setDemoSelected : setLiveSelected;
const [editing, setEditing] = useState<"new" | "edit" | "duplicate" | null>(null);
const [batchId, setBatchId] = useState("latest");
const [historyOffset, setHistoryOffset] = useState(0);
@ -146,7 +154,7 @@ export function LensView({ accessToken, readOnly = false }: { accessToken: strin
queryKey: ["lens-activity-available", accessToken],
queryFn: () => apiClient.get<{ traces: boolean; requests: boolean }>("/lens/activity/available", { accessToken }),
enabled: loaded,
refetchInterval: 5000,
refetchInterval: demo ? (false as const) : 5000,
};
const activity = useQuery(activityOptions);
const tracesReady = activity.data?.traces === true && !activity.error;
@ -157,7 +165,7 @@ export function LensView({ accessToken, readOnly = false }: { accessToken: strin
queryKey: ["lens-history", lens?.id, historyOffset, accessToken],
enabled: !!lens,
queryFn: () => apiClient.get<Job[]>(`/lens/${lens?.id}/runs`, { accessToken, query: { offset: historyOffset } }),
refetchInterval: 10000,
refetchInterval: demo ? (false as const) : 10000,
};
const history = useQuery(historyQuery);
const historical = useQuery({
@ -297,7 +305,7 @@ export function LensView({ accessToken, readOnly = false }: { accessToken: strin
}}
onConnect={() => setWorkerSetup(true)}
onCreate={() => setEditing("new")}
onExample={() => setShowExample(true)}
onDemo={activity.isSuccess && !ready ? onDemo : undefined}
/>
)}
{showReadiness && !ready && (
@ -687,7 +695,6 @@ export function LensView({ accessToken, readOnly = false }: { accessToken: strin
}
/>
)}
{showExample && <InvestigationExample onClose={() => setShowExample(false)} />}
{monitoring && lens && (
<MonitoringSetup
settings={lens.settings}

13
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/LensWelcome.tsx
View file

@ -1,6 +1,7 @@
import Link from "next/link";
import { Check, ArrowRight, Loader2 } from "lucide-react";
import { Check, Loader2 } from "lucide-react";
import { Button, buttonVariants } from "@/components/ui/button";
import { LensPreviewButton } from "@/components/lens/LensPreviewButton";
import { uiHref } from "@/utils/uiHref";
export function LensWelcome({
@ -13,7 +14,7 @@ export function LensWelcome({
onConnect,
onCreate,
onRetry,
onExample,
onDemo,
}: {
tracesReady: boolean;
requestsReady?: boolean;
@ -24,7 +25,7 @@ export function LensWelcome({
onConnect: () => void;
onCreate: () => void;
onRetry: () => void;
onExample: () => void;
onDemo?: () => void;
}) {
const activityReady = tracesReady || requestsReady;
const workerReady = activityReady && connected;
@ -39,16 +40,14 @@ export function LensWelcome({
className: "col-start-2 w-fit sm:col-start-auto",
});
return (
<section aria-labelledby="lens-welcome" className="max-w-3xl py-4 sm:py-6">
<section aria-labelledby="lens-welcome" className="max-w-3xl pb-6">
{onDemo && <LensPreviewButton onClick={onDemo} />}
<h2 id="lens-welcome" className="text-xl font-semibold tracking-tight">
Find what needs attention
</h2>
<p className="mt-2 max-w-lg text-sm leading-6 text-muted-foreground">
Check how your agents behave. Get findings you can trace back to what happened.
</p>
<Button variant="link" className="mt-3 h-auto px-0 text-sm" onClick={onExample}>
View an example <ArrowRight className="size-3.5" />
</Button>
<ol className="mt-8 divide-y border-y">
<li className="grid grid-cols-[28px_minmax(0,1fr)] items-center gap-x-4 gap-y-3 py-5 sm:grid-cols-[28px_minmax(0,1fr)_auto]">
<Step number={1} complete={activityReady} checking={checking} active={!activityReady} />

137
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/LensWorkspace.integration.test.tsx Normal file
View file

@ -0,0 +1,137 @@
import { fireEvent, screen, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { chooseSelectOption, renderWithProviders, testQueryClient } from "@/../tests/test-utils";
import { LensWorkspace } from "./LensWorkspace";
import { createLensDemoData } from "./lensDemoData";
const network = vi.fn<typeof fetch>();
beforeEach(() => {
testQueryClient.clear();
vi.stubGlobal("fetch", network);
network.mockReset();
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/v1/traces") return Response.json({ detail: "Tracing is not enabled" }, { status: 501 });
if (path === "/lens") return Response.json({ lenses: [], workers: [], tracing_enabled: false });
return Response.json({ data: [], traces: false, requests: false });
});
});
describe("Lens interactive demo", () => {
it("opens without tracing, filters sample runs, and restores the live view without mixing data", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Internal User" readOnly={false} />, {
onUrlUpdate,
});
expect(await screen.findByText("Enable tracing")).toBeVisible();
await user.click(screen.getByRole("button", { name: "Preview sample" }));
expect(await screen.findByText("Where is order #1042?")).toBeVisible();
expect(screen.getByText("You’re viewing demo data")).toBeVisible();
expect(screen.queryByRole("button", { name: "Set up tracing" })).not.toBeInTheDocument();
network.mockClear();
await user.click(screen.getByRole("button", { name: "Refresh", exact: true }));
expect(screen.getByText("Where is order #1042?")).toBeVisible();
fireEvent.change(screen.getByPlaceholderText("Search input or trace ID"), { target: { value: "headphones" } });
expect(screen.getByText("Can I return my headphones?")).toBeVisible();
expect(screen.queryByText("Where is order #1042?")).not.toBeInTheDocument();
fireEvent.change(screen.getByRole("textbox", { name: "Search runs" }), { target: { value: "" } });
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Filter by agent" }), "support_agent");
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Filter by status" }), "Failed");
expect(within(screen.getByRole("table", { name: "Agent runs" })).getAllByRole("row")).toHaveLength(4);
await user.click(screen.getByRole("tab", { name: "Investigations", exact: true }));
expect(await screen.findByRole("button", { name: /Support quality/ })).toBeVisible();
expect(screen.queryByRole("button", { name: "New investigation" })).not.toBeInTheDocument();
expect(network).not.toHaveBeenCalled();
expect(onUrlUpdate).not.toHaveBeenCalled();
await user.click(screen.getByRole("button", { name: "Exit demo" }));
expect(await screen.findByText("Enable tracing")).toBeVisible();
expect(screen.queryByText("You’re viewing demo data")).not.toBeInTheDocument();
expect(screen.queryByText("Can I return my headphones?")).not.toBeInTheDocument();
});
it("connects findings and history to their original trace without live requests or URL changes", async () => {
const user = userEvent.setup();
const onUrlUpdate = vi.fn();
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, {
searchParams: "?tab=investigations",
onUrlUpdate,
});
await screen.findByRole("button", { name: "Preview sample" });
await user.click(screen.getByRole("button", { name: "Preview sample" }));
await user.click(await screen.findByRole("button", { name: /Support quality/ }));
network.mockClear();
await user.click(screen.getByRole("button", { name: /Repeated lookups leave customers without an answer/ }));
const finding = screen.getByRole("dialog");
expect(within(finding).getByText(/The support agent retries/)).toBeVisible();
const summaries = within(finding).getAllByText("support_agent", { exact: true });
await user.click(summaries[0]);
await user.click(within(finding).getAllByRole("button", { name: /Open original step/ })[0]);
expect(await screen.findByRole("complementary", { name: "Span details" })).toHaveTextContent(
"I will check that for you.",
);
await user.click(screen.getByRole("button", { name: "Copy for agent" }));
expect(await navigator.clipboard.readText()).toContain("I will check that for you.");
expect(await navigator.clipboard.readText()).not.toContain("Authorization");
await user.click(screen.getByRole("tab", { name: "Attributes" }));
expect(await screen.findByText("gen_ai.agent.name")).toBeVisible();
await user.click(
within(screen.getByRole("dialog", { name: "Original run" })).getByRole("button", { name: "Close", exact: true }),
);
await user.click(within(screen.getByRole("dialog")).getByRole("button", { name: "Close", exact: true }));
await user.click(screen.getByRole("tab", { name: "History" }));
await user.click(screen.getAllByRole("button", { name: /runs reviewed/ })[1]);
expect(await screen.findByText(/1 linked run · high priority/)).toBeVisible();
expect(network).not.toHaveBeenCalled();
expect(onUrlUpdate).not.toHaveBeenCalled();
await user.click(screen.getByRole("button", { name: "Exit demo" }));
expect(await screen.findByRole("heading", { name: "Find what needs attention" })).toBeVisible();
});
it("has no demo entry for existing investigations, populated traces, or connecting another agent", async () => {
const user = userEvent.setup();
const data = createLensDemoData();
const saved = data.lenses[0];
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [saved], workers: [], tracing_enabled: true });
if (path.endsWith("/runs")) return Response.json(saved.jobs);
if (path === "/v1/traces") return Response.json({ data: data.runs.map((run) => run.trace.summary) });
return Response.json({ data: [], traces: true, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />, {
searchParams: `?tab=investigations&lens=${saved.id}`,
});
expect(await screen.findByRole("heading", { name: saved.settings.name })).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
await user.click(within(screen.getByRole("tablist", { name: "Lens" })).getByRole("tab", { name: "Traces" }));
expect(await screen.findByText("Where is order #1042?")).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Set up tracing" }));
expect(await screen.findByRole("heading", { name: "Connect another agent" })).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
});
it("shows the header preview only for the active tab that still needs setup", async () => {
const user = userEvent.setup();
const saved = createLensDemoData().lenses[0];
network.mockImplementation(async (input) => {
const path = new URL(String(input), "http://localhost").pathname;
if (path === "/lens") return Response.json({ lenses: [saved], workers: [], tracing_enabled: false });
if (path.endsWith("/runs")) return Response.json(saved.jobs);
if (path === "/v1/traces") return Response.json({ detail: "Tracing is not enabled" }, { status: 501 });
return Response.json({ data: [], traces: false, requests: false });
});
renderWithProviders(<LensWorkspace accessToken="live-token" userRole="Admin" readOnly={false} />);
expect(await screen.findByRole("button", { name: "Preview sample" })).toBeVisible();
const tabs = within(screen.getByRole("tablist", { name: "Lens" }));
await user.click(tabs.getByRole("tab", { name: "Investigations" }));
expect(await screen.findByRole("button", { name: new RegExp(saved.settings.name) })).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
await user.click(tabs.getByRole("tab", { name: "Traces" }));
await user.click(await screen.findByRole("button", { name: "Preview sample" }));
expect(await screen.findByRole("table", { name: "Agent runs" })).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
});
});

117
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/LensWorkspace.tsx Normal file
View file

@ -0,0 +1,117 @@
"use client";
import { useEffect, useState } from "react";
import { Aperture } from "lucide-react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { parseAsString, parseAsStringLiteral, useQueryState } from "nuqs";
import AgentTracesPage from "@/components/view_logs/TraceView/AgentTracesPage";
import { DemoNotice } from "@/components/shared/DemoNotice";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { LensDemoContext, useLensDemo } from "@/components/lens/LensDemoContext";
import { LensPreviewTarget } from "@/components/lens/LensPreviewButton";
import { isProxyAdminRole, isProxyAdminTierRole } from "@/utils/roles";
import { LensView } from "./LensView";
import { createLensDemo } from "./lensDemoData";
type Tab = "traces" | "investigations";
type WorkspaceProps = { accessToken: string; userRole: string; readOnly: boolean };
export function LensWorkspace(props: WorkspaceProps) {
const [demoTab, setDemoTab] = useState<Tab | null>(null);
return demoTab ? (
<DemoSession initialTab={demoTab} onExit={() => setDemoTab(null)} />
) : (
<LensContent {...props} onDemo={setDemoTab} />
);
}
function DemoSession({ initialTab, onExit }: { initialTab: Tab; onExit: () => void }) {
const [demo] = useState(createLensDemo);
const [client] = useState(
() => new QueryClient({ defaultOptions: { queries: { retry: false, staleTime: Infinity } } }),
);
useEffect(
() => () => {
client.clear();
},
[client],
);
return (
<LensDemoContext.Provider value={demo}>
<QueryClientProvider client={client}>
<LensContent accessToken="lens-demo" userRole="" readOnly initialTab={initialTab} onExit={onExit} />
</QueryClientProvider>
</LensDemoContext.Provider>
);
}
function LensContent({
accessToken,
userRole,
readOnly,
initialTab = "traces",
onDemo,
onExit,
}: WorkspaceProps & { initialTab?: Tab; onDemo?: (tab: Tab) => void; onExit?: () => void }) {
const demo = useLensDemo();
const [tab, setTab] = useQueryState(
"tab",
parseAsStringLiteral(["traces", "investigations"]).withOptions({ history: "push" }),
);
const [lensId] = useQueryState("lens", parseAsString);
const [demoTab, setDemoTab] = useState(initialTab);
const [previewTarget, setPreviewTarget] = useState<HTMLDivElement | null>(null);
const defaultTab = lensId ? "investigations" : "traces";
const activeTab = demo ? demoTab : tab ?? defaultTab;
const openDemo = onDemo ? () => onDemo(activeTab) : undefined;
return (
<LensPreviewTarget.Provider value={previewTarget}>
<main className="flex w-full min-w-0 flex-1 flex-col gap-5 p-6 md:p-8">
<div className="flex min-h-9 flex-wrap items-center justify-between gap-3">
<h1 className="flex items-center gap-2 text-2xl font-semibold tracking-tight">
<Aperture aria-hidden="true" className="size-7" strokeWidth={1.75} />
Lens
</h1>
<div ref={setPreviewTarget} />
</div>
{demo && <DemoNotice onExit={onExit} />}
<Tabs
value={activeTab}
onValueChange={(value) => (demo ? setDemoTab(value as Tab) : void setTab(value as Tab))}
className="min-h-0 flex-1 gap-4"
>
<TabsList variant="line" aria-label="Lens" className="w-full justify-start gap-6 border-b px-0">
<TabsTrigger value="traces" className="flex-none px-0">
Traces
</TabsTrigger>
<TabsTrigger value="investigations" className="flex-none px-0">
Investigations
</TabsTrigger>
</TabsList>
<TabsContent value="traces" keepMounted className="min-h-0">
<AgentTracesPage
accessToken={accessToken}
isActive={activeTab === "traces"}
readOnly={readOnly}
canMintTracingKey={!demo && isProxyAdminRole(userRole)}
onDemo={activeTab === "traces" ? openDemo : undefined}
/>
</TabsContent>
<TabsContent value="investigations" keepMounted={!!demo}>
{demo || isProxyAdminTierRole(userRole) ? (
<LensView
accessToken={accessToken}
readOnly={readOnly || !isProxyAdminRole(userRole)}
onDemo={activeTab === "investigations" ? openDemo : undefined}
/>
) : (
<p className="py-6 text-sm text-muted-foreground">
Investigations require proxy administrator access. You can still view your traces.
</p>
)}
</TabsContent>
</Tabs>
</main>
</LensPreviewTarget.Provider>
);
}

2
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/WorkerSetup.tsx
View file

@ -21,7 +21,7 @@ import { AnalysisKey, AnalysisAccessFields, createAnalysisKey, type AnalysisAcce
import type { LensList, WorkerCreated } from "./lensData";
export const LENS_WORKER_IMAGE =
"ghcr.io/berriai/litellm-lens-worker@sha256:67eba741c1b97c749975c5c38e2370a603e1105babc908d613c1b79d7b995393";
"ghcr.io/berriai/litellm-lens-worker@sha256:44f0597c7583dcfef999ece9a8bc02cfeb9f0f5167a1221cee3bd10b1b79271b";
function initialProxyAddress(): string {
const url = new URL(proxyBaseUrl || serverRootPath, window.location.origin);

116
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/lensData.test.ts
View file

@ -4,6 +4,10 @@ import {
workerConnected,
type LensList,
analysisProgress,
analysisFraction,
analysisPace,
remainingLabel,
stageDurations,
normalizeFilters,
sortedFindings,
type Finding,
@ -58,6 +62,28 @@ const job: Job = {
};
describe("Analysis progress", () => {
it.each(["Collecting executions", "Reading executions"])(
"does not turn an unreported run count into zero runs during %s",
(stage) => {
expect(analysisProgress({ ...job, stage })).toEqual({
step: -1,
title: "Preparing activity",
done: 0,
total: 0,
detail: "Loading the runs selected for this investigation.",
});
},
);
it("shows the selected count as soon as the worker reports it", () => {
expect(analysisProgress({ ...job, coverage: { ...coverage, selected: 7 } })).toMatchObject({
step: 0,
done: 0,
total: 7,
detail: "0 of 7 selected runs reviewed",
});
});
it("measures review progress against the sample, not all eligible runs", () => {
const expected = { step: 0, done: 7, total: 20, detail: "7 of 20 selected runs reviewed" };
expect(
@ -114,6 +140,96 @@ describe("Analysis progress", () => {
});
});
describe("Analysis pace", () => {
it("fills the bar left to right across stages without jumping backwards at a stage boundary", () => {
const endOfReview = analysisFraction({ step: 0, done: 20, total: 20 });
const startOfGrouping = analysisFraction({ step: 1, done: 0, total: 4 });
expect(analysisFraction({ step: -1, done: 0, total: 0 })).toBe(0);
expect(analysisFraction({ step: 0, done: 10, total: 20 })).toBeLessThan(endOfReview);
expect(startOfGrouping).toBeCloseTo(endOfReview);
expect(analysisFraction({ step: 2, done: 5, total: 5 })).toBeCloseTo(1);
});
it("measures speed within the current stage and projects time left from overall progress", () => {
const start = Date.parse("2026-09-30T12:00:00Z");
const pace = analysisPace(
[
{ at: start, step: 0, done: 0, fraction: 0 },
{ at: start + 30000, step: 0, done: 30, fraction: 0.25 },
],
start + 30000,
);
expect(pace.perMinute).toBe(60);
expect(pace.secondsLeft).toBe(90);
});
it("waits for enough samples instead of showing a wild first estimate", () => {
const start = Date.parse("2026-09-30T12:00:00Z");
expect(analysisPace([{ at: start, step: 0, done: 1, fraction: 0.01 }], start + 2000)).toEqual({
perMinute: null,
secondsLeft: null,
});
expect(remainingLabel(null)).toBe("estimating");
});
it("lengthens the estimate while progress stalls", () => {
const start = Date.parse("2026-09-30T12:00:00Z");
const samples = [
{ at: start, step: 0, done: 0, fraction: 0 },
{ at: start + 10000, step: 0, done: 10, fraction: 0.1 },
];
const moving = analysisPace(samples, start + 10000).secondsLeft ?? 0;
const stalled = analysisPace(samples, start + 40000).secondsLeft ?? 0;
expect(stalled).toBeGreaterThan(moving);
});
it("still estimates when the worker reports progress less than once a minute", () => {
const start = Date.parse("2026-09-30T12:00:00Z");
const pace = analysisPace(
[
{ at: start, step: 2, done: 0, fraction: 0.8 },
{ at: start + 90000, step: 2, done: 1, fraction: 0.85 },
],
start + 90000,
);
expect(pace.secondsLeft).toBeCloseTo(270);
expect(pace.perMinute).toBeCloseTo(2 / 3);
});
it("measures from the last minute rather than the whole run once updates are frequent", () => {
const start = Date.parse("2026-09-30T12:00:00Z");
const samples = [
{ at: start, step: 0, done: 0, fraction: 0 },
{ at: start + 120000, step: 0, done: 12, fraction: 0.06 },
{ at: start + 180000, step: 0, done: 72, fraction: 0.36 },
];
expect(analysisPace(samples, start + 180000).perMinute).toBe(60);
});
it("rounds remaining time up so the label never promises less than the estimate", () => {
expect(remainingLabel(61)).toBe("~2m");
expect(remainingLabel(30)).toBe("<1m");
});
});
describe("Stage durations", () => {
const start = Date.parse("2026-09-30T12:00:00Z");
const createdAt = "2026-09-30T12:00:00Z";
it("times finished stages from the transitions it saw and the active stage up to now", () => {
const samples = [
{ at: start + 5000, step: 0, done: 10, fraction: 0.1 },
{ at: start + 124000, step: 1, done: 0, fraction: 0.6 },
];
expect(stageDurations(samples, createdAt, start + 145000)).toEqual([124, 21, null]);
});
it("does not guess when a stage started before the page was opened", () => {
const samples = [{ at: start + 90000, step: 1, done: 2, fraction: 0.7 }];
expect(stageDurations(samples, createdAt, start + 100000)).toEqual([null, null, null]);
});
});
describe("Lens selection and findings", () => {
it("preserves literal equals signs in a metadata value", () => {
expect(normalizeFilters([{ key: " swarm ", value: " research=v2 " }])).toEqual([

93
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/lensData.ts
View file

@ -108,6 +108,15 @@ export function analysisProgress(job: Job) {
: `${investigated} patterns checked against the original activity`,
};
}
if (!selected) {
return {
step: -1,
title: "Preparing activity",
done: 0,
total: 0,
detail: "Loading the runs selected for this investigation.",
};
}
return {
step: 0,
title: "Reviewing activity",
@ -117,8 +126,90 @@ export function analysisProgress(job: Job) {
};
}
export function analysisStages(job: Job): { done: number; total: number }[] {
const {
screened = 0,
selected = 0,
grouped_batches = 0,
grouping_batches = 0,
investigated = 0,
candidates = 0,
} = job.coverage ?? {};
return [
{ done: screened, total: selected },
{ done: grouped_batches, total: grouping_batches },
{ done: investigated, total: candidates },
];
}
export interface ProgressSample {
at: number;
step: number;
done: number;
fraction: number;
}
export const stageWeights = [0.6, 0.2, 0.2];
export function analysisFraction({
step,
done,
total,
}: Pick<ReturnType<typeof analysisProgress>, "step" | "done" | "total">): number {
if (step < 0) return 0;
const before = stageWeights.slice(0, step).reduce((sum, weight) => sum + weight, 0);
return before + stageWeights[step] * (total ? Math.min(1, done / total) : 0);
}
function windowStart(samples: readonly ProgressSample[], now: number): ProgressSample | undefined {
return samples.findLast((sample) => now - sample.at >= 60000) ?? samples[0];
}
export function analysisPace(samples: readonly ProgressSample[], now: number) {
const latest = samples.at(-1);
if (!latest) return { perMinute: null, secondsLeft: null };
const first = windowStart(
samples.filter((sample) => sample.step === latest.step),
now,
);
const anchor = windowStart(samples, now);
if (!first || !anchor) return { perMinute: null, secondsLeft: null };
const stepMinutes = (now - first.at) / 60000;
const perMinute = stepMinutes >= 1 / 6 ? (latest.done - first.done) / stepMinutes : null;
const spanSeconds = (now - anchor.at) / 1000;
const gained = latest.fraction - anchor.fraction;
const secondsLeft = spanSeconds >= 10 && gained > 0 ? ((1 - latest.fraction) * spanSeconds) / gained : null;
return { perMinute, secondsLeft };
}
export function stageDurations(samples: readonly ProgressSample[], createdAt: string, now: number): (number | null)[] {
const current = samples.at(-1)?.step ?? -1;
const starts = [0, 1, 2].map((stage) => {
if (stage === 0) return Date.parse(createdAt);
const entered = samples.findIndex(
(sample, index) => index > 0 && sample.step >= stage && samples[index - 1].step < stage,
);
return entered < 0 ? null : samples[entered].at;
});
return starts.map((start, stage) => {
if (start === null || stage > current) return null;
const end = stage === current ? now : starts[stage + 1];
return end === null ? null : Math.max(0, Math.floor((end - start) / 1000));
});
}
export function remainingLabel(seconds: number | null): string {
if (seconds === null) return "estimating";
if (seconds < 60) return "<1m";
if (seconds < 3600) return `~${Math.ceil(seconds / 60)}m`;
return `~${Math.floor(seconds / 3600)}h ${Math.ceil((seconds % 3600) / 60)}m`;
}
export function analysisElapsed(createdAt: string, now: number): string {
const seconds = Math.max(0, Math.floor((now - Date.parse(createdAt)) / 1000));
return durationText(Math.max(0, Math.floor((now - Date.parse(createdAt)) / 1000)));
}
export function durationText(seconds: number): string {
if (!Number.isFinite(seconds)) return "0s";
if (seconds < 60) return `${seconds}s`;
if (seconds < 3600) return `${Math.floor(seconds / 60)}m ${seconds % 60}s`;

55
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/lensDemoData.test.ts Normal file
View file

@ -0,0 +1,55 @@
import { describe, expect, it, vi } from "vitest";
import { createLensDemo, createLensDemoData } from "./lensDemoData";
import type { TracePage } from "@/components/view_logs/TraceView/traceTypes";
import { evidenceTarget } from "./lensData";
describe("Lens demo data", () => {
it("links every finding to the quoted original step and assessed run", () => {
const data = createLensDemoData();
for (const lens of data.lenses) {
for (const job of lens.jobs) {
for (const finding of job.findings ?? []) {
for (const evidence of finding.evidence) {
const target = evidenceTarget(evidence.execution_id);
const run = data.runs.find(({ trace }) => trace.summary.trace_id === target?.id);
const step = run?.details.find((span) => span.span_id === evidence.span_id);
expect(step?.output).toContain(evidence.quote);
expect(job.sample?.executions.map((execution) => execution.id)).toContain(evidence.execution_id);
expect(
job.assessments.find((assessment) => assessment.execution_id === evidence.execution_id)?.[
finding.kind === "issue" ? "issue_checks" : "pattern_checks"
],
).toContain(finding.check_id);
}
}
}
}
});
it("keeps trace totals, timestamps, and agent names consistent", () => {
const data = createLensDemoData();
for (const { trace } of data.runs) {
expect(trace.summary.span_count).toBe(trace.spans.length);
expect(trace.summary.agent_names).toContain(trace.agents[0].name);
expect(trace.summary.error_count).toBe(trace.spans.filter((span) => span.status === "error").length);
for (const span of trace.spans) {
expect(span.start_offset_ms + span.duration_ms).toBeLessThanOrEqual(trace.summary.duration_ms);
}
}
});
it("filters time windows locally and rejects writes or unknown reads without network access", async () => {
const network = vi.spyOn(globalThis, "fetch");
const now = Date.now();
const { client } = createLensDemo(now);
const all = await client.get<TracePage>("/v1/traces");
const recent = await client.get<TracePage>("/v1/traces", { query: { start_ms: now - 3600_000, end_ms: now } });
expect(recent.data.length).toBeGreaterThan(0);
expect(recent.data.length).toBeLessThan(all.data.length);
expect(recent.data.every((trace) => Date.parse(trace.start_time) >= now - 3600_000)).toBe(true);
await expect(client.post("/lens", { body: {} })).rejects.toMatchObject({ status: 403 });
await expect(client.get("/lens/real-investigation")).rejects.toMatchObject({ status: 404 });
expect(network).not.toHaveBeenCalled();
network.mockRestore();
});
});

483
ui/litellm-dashboard/src/app/(dashboard)/lens/_components/lensDemoData.ts Normal file
View file

@ -0,0 +1,483 @@
import { createApiClient } from "@/lib/http/client";
import type { LensDemo } from "@/components/lens/LensDemoContext";
import type { Trace, Span, SpanDetail } from "@/components/view_logs/TraceView/traceTypes";
import type { Lens, Finding, Job, Settings } from "./lensData";
type Scenario = {
agent: string;
question: string;
tool: string;
result: string;
answer: string;
failed?: boolean;
};
const scenarios: Scenario[] = [
{
agent: "support_agent",
question: "Where is order #1042?",
tool: "lookup_order",
result: "Order service timed out",
answer: "I will check that for you.",
failed: true,
},
{
agent: "research_agent",
question: "Which database should we use for analytics?",
tool: "search_docs",
result: "Column stores are designed for analytical queries. No comparative benchmark is available.",
answer: "The column store is 40% faster than the row store for your workload.",
},
{
agent: "support_agent",
question: "Can I return my headphones?",
tool: "lookup_return_policy",
result: "Returns are accepted within 30 days. This order was delivered 10 days ago.",
answer: "Yes. Your order is eligible for a return. I can help you start it.",
},
{
agent: "release_agent",
question: "Is the checkout release ready to ship?",
tool: "read_test_results",
result: "Checkout: 120 passed, 0 failed. Rollback plan approved.",
answer: "All 120 checkout tests passed and the rollback plan is approved. The release is ready.",
},
{
agent: "support_agent",
question: "My order #1088 has not arrived",
tool: "lookup_order",
result: "Order service timed out",
answer: "I will check that for you.",
failed: true,
},
{
agent: "research_agent",
question: "How long do we retain audit logs?",
tool: "search_docs",
result: "Audit log retention is 90 days on the current plan.",
answer: "Audit logs are retained for 90 days, according to the plan documentation.",
},
{
agent: "release_agent",
question: "Review the search release",
tool: "read_test_results",
result: "Search: 86 passed, 1 failed. Unicode query regression remains open.",
answer: "Hold the release. The Unicode query regression is still failing.",
},
{
agent: "support_agent",
question: "Update the address on order #1091",
tool: "lookup_order",
result: "Order #1091 has not shipped. Address changes are allowed.",
answer: "Your order has not shipped. Please confirm the new delivery address.",
},
{
agent: "research_agent",
question: "Can this plan store data in Europe?",
tool: "search_docs",
result: "Regional availability is not documented for this plan.",
answer: "I could not verify European data residency for this plan. Please confirm with support before choosing it.",
},
{
agent: "support_agent",
question: "Check the delivery date for order #1107",
tool: "lookup_order",
result: "Order service timed out",
answer: "The order service is unavailable. I can connect you to support instead.",
failed: true,
},
{
agent: "release_agent",
question: "Summarize the authentication release checks",
tool: "read_test_results",
result: "Authentication: 214 passed, 0 failed. Security review approved.",
answer: "All 214 tests passed and the security review is approved.",
},
{
agent: "research_agent",
question: "Does the API support request retries?",
tool: "search_docs",
result: "Retry requests with exponential backoff after a 429 or 503 response.",
answer: "Yes. Use exponential backoff for 429 and 503 responses, as documented.",
},
];
const executionId = (traceId: string) => btoa(JSON.stringify(["traces", "", traceId]));
const iso = (time: number) => new Date(time).toISOString();
function makeTrace(scene: Scenario, index: number, now: number) {
const traceId = (index + 1).toString(16).padStart(32, "0");
const spanId = (step: number) => `${index + 1}${step}`.padStart(16, "0");
const toolCount = scene.failed && index !== 9 ? 3 : 1;
const duration = toolCount === 3 ? 6340 : 3180 + index * 137;
const base: Span = {
agent: scene.agent,
duration_ms: duration,
error: null,
error_truncated: false,
framework: "",
input_preview: scene.question,
input_tokens: 0,
output_tokens: 0,
litellm_request_id: null,
model: null,
name: scene.agent,
parent_span_id: null,
span_id: spanId(0),
spend: null,
start_offset_ms: 0,
status: "ok",
type: "agent",
};
const tools: Span[] = Array.from({ length: toolCount }, (_, i) => ({
...base,
span_id: spanId(i + 1),
parent_span_id: base.span_id,
name: scene.tool,
type: "tool",
start_offset_ms: 150 + i * 1600,
duration_ms: scene.failed ? 1500 : 340,
status: scene.failed ? "error" : "ok",
error: scene.failed ? scene.result : null,
}));
const model: Span = {
...base,
span_id: spanId(5),
parent_span_id: base.span_id,
name: "Generate response",
type: "llm",
model: "demo-chat-model",
start_offset_ms: toolCount * 1600,
duration_ms: 1200,
input_tokens: 520 + index * 41,
output_tokens: 48 + index * 7,
spend: 0.003 + index * 0.0002,
};
const trace: Trace = {
summary: {
agent_count: 1,
agent_invocations: 1,
agent_names: [scene.agent],
duration_ms: duration,
error_count: scene.failed ? toolCount : 0,
frameworks: [],
input_preview: scene.question,
input_tokens: model.input_tokens,
output_tokens: model.output_tokens,
llm_calls: 1,
models: [model.model!],
name: scene.agent,
service: "demo-agents",
span_count: toolCount + 2,
spend: model.spend,
start_time: iso(now - (index + 1) * 35 * 60_000),
status: scene.failed ? "error" : "ok",
tool_calls: toolCount,
trace_id: traceId,
},
agents: [
{
name: scene.agent,
parent_agent: null,
duration_ms: duration,
invocations: 1,
llm_calls: 1,
tool_calls: toolCount,
spend: model.spend,
},
],
spans: [base, ...tools, model],
};
const details: SpanDetail[] = trace.spans.map((span) => ({
span_id: span.span_id,
input:
span.type === "tool"
? JSON.stringify({ query: scene.question })
: JSON.stringify([{ role: "user", content: scene.question }]),
output: span.type === "tool" ? scene.result : JSON.stringify([{ role: "assistant", content: scene.answer }]),
attributes: {
"gen_ai.agent.name": scene.agent,
"service.name": "demo-agents",
demo: "true",
...(span.model ? { "gen_ai.request.model": span.model } : {}),
},
}));
return { trace, details };
}
export function createLensDemoData(now = Date.now()) {
const runs = scenarios.map((scene, index) => makeTrace(scene, index, now));
const finding = ({
id,
check,
title,
description,
suggestion,
indices,
kind = "issue",
}: {
id: string;
check: string;
title: string;
description: string;
suggestion: string;
indices: number[];
kind?: Finding["kind"];
}): Finding => ({
id,
check_id: check,
title,
description,
suggestion,
kind,
priority: kind === "issue" ? "high" : "low",
status: "open",
reason: "",
revision: 1,
first_seen: iso(now - 86_400_000),
last_seen: iso(now - 300_000),
limitation: "",
occurrences: indices.map((i) => executionId(runs[i].trace.summary.trace_id)),
evidence: indices.map((i) => ({
execution_id: executionId(runs[i].trace.summary.trace_id),
span_id: runs[i].trace.spans.at(-1)!.span_id,
quote: scenarios[i].answer,
role: "support",
})),
});
const findingInputs: Parameters<typeof finding>[0][] = [
{
id: "failed-lookups",
check: "recover",
title: "Repeated lookups leave customers without an answer",
description:
"The support agent retries the same unavailable order service three times, then promises to check without answering or offering a handoff.",
suggestion: "After repeated failures, explain the problem and offer a handoff.",
indices: [0, 4],
},
{
id: "safe-handoff",
check: "recover",
title: "A clear handoff helps when the order service is unavailable",
description:
"The agent explains the service outage and offers a support handoff instead of promising an answer it cannot provide.",
suggestion: "Keep this fallback for unavailable services.",
indices: [9],
kind: "pattern",
},
{
id: "unsupported-claim",
check: "grounding",
title: "Performance claim has no supporting benchmark",
description:
"The answer claims a 40% performance improvement, but the retrieved documentation contains no comparative benchmark.",
suggestion: "Require a benchmark source for numeric performance claims, or remove the comparison.",
indices: [1],
},
{
id: "uncertainty",
check: "grounding",
title: "Missing information is acknowledged",
description:
"When documentation does not establish regional availability, the agent says so and asks the user to verify it.",
suggestion: "Keep stating when a source does not answer the question.",
indices: [8],
kind: "pattern",
},
{
id: "release-blocked",
check: "release",
title: "Failing checks correctly block the release",
description: "The release agent identifies an unresolved regression and recommends holding the release.",
suggestion: "Continue requiring passing checks before recommending a release.",
indices: [6],
kind: "pattern",
},
];
const findings = findingInputs.map(finding);
const definitions = [
{
id: "support",
name: "Support quality",
agent: "support_agent",
check: "recover",
context:
"Answer the customer's question using order information. If a tool fails, explain the problem and offer a handoff.",
instruction: "Look for repeated failed calls and conversations that end without an answer or a handoff.",
},
{
id: "research",
name: "Research accuracy",
agent: "research_agent",
check: "grounding",
context: "Answer questions using verified documentation. Acknowledge missing information.",
instruction: "Find claims that are not supported by the retrieved sources.",
},
{
id: "release",
name: "Release readiness",
agent: "release_agent",
check: "release",
context: "Review test results and recommend a release only when all required checks pass.",
instruction: "Check whether failed tests are acknowledged before a release recommendation.",
},
];
const lenses: Lens[] = definitions.map((definition) => {
const settings: Settings = {
name: definition.name,
agent_name: definition.agent,
context: definition.context,
checks: [{ id: definition.check, instruction: definition.instruction, enabled: true }],
source: "traces",
service: "",
filters: [],
team_id: "",
execution_ids: [],
lookback_hours: 24,
sample_size: 0,
sample_percent: 100,
concurrency: 8,
monthly_budget: 100,
interval_minutes: 30,
enabled: false,
model: "demo-analysis-model",
};
const executions = runs
.filter(({ trace }) => trace.summary.name === definition.agent)
.map(({ trace }) => ({
id: executionId(trace.summary.trace_id),
trace_ref: "",
metadata: [],
root_seen: true,
service: trace.summary.service,
source: "traces" as const,
trace_id: trace.summary.trace_id,
team_id: "",
name: trace.summary.name,
start_time: trace.summary.start_time,
span_count: trace.summary.span_count,
}));
const relevant = findings.filter((f) => f.check_id === definition.check);
const jobs: Job[] = [0, 1].map((day) => {
const sample = day === 0 ? executions : executions.slice(1);
const selectedIds = new Set(sample.map((item) => item.id));
const snapshot = relevant
.map((f) => ({
...f,
occurrences: f.occurrences.filter((id) => selectedIds.has(id)),
evidence: f.evidence.filter((e) => selectedIds.has(e.execution_id)),
}))
.filter((f) => f.occurrences.length > 0);
return {
id: `${definition.id}-scan-${day}`,
findings: snapshot,
settings,
revision: 1,
assessments: sample.map((e) => ({
execution_id: e.id,
cannot_assess: false,
issue_checks: snapshot.some((f) => f.kind === "issue" && f.occurrences.includes(e.id))
? [definition.check]
: [],
pattern_checks: snapshot.some((f) => f.kind === "pattern" && f.occurrences.includes(e.id))
? [definition.check]
: [],
})),
attempts: 1,
error: "",
cost: sample.length * 0.012,
coverage: {
eligible: sample.length,
selected: sample.length,
screened: sample.length,
investigated: sample.length,
inconclusive: 0,
grouping_batches: 1,
grouped_batches: 1,
candidates: snapshot.length,
partial: 0,
unassessable: 0,
},
status: "completed",
stage: "Complete",
created_at: iso(now - 330_000 - day * 60_000),
finished_at: iso(now - 300_000 - day * 60_000),
start: iso(now - 86_400_000),
end: iso(now - 300_000 - day * 60_000),
sample: { eligible: sample.length, selected: sample.length, executions: sample },
};
});
return {
id: definition.id,
version: 1,
revision: 1,
spent: jobs.reduce((sum, job) => sum + job.cost, 0),
scope: { all_teams: true, api_key_hash: "", team_id: "" },
settings,
created_at: iso(now - 2 * 86_400_000),
next_run_at: iso(now),
budget_month: iso(now).slice(0, 7),
findings: relevant,
jobs,
};
});
return { runs, lenses };
}
function readDemoTrace(data: ReturnType<typeof createLensDemoData>, url: URL): unknown {
const parts = url.pathname.split("/").filter(Boolean);
if (parts[0] !== "v1" || parts[1] !== "traces") return undefined;
if (!parts[2]) {
const start = Number(url.searchParams.get("start_ms") ?? 0);
const end = Number(url.searchParams.get("end_ms") ?? Infinity);
return {
data: data.runs
.map((run) => run.trace.summary)
.filter((trace) => Date.parse(trace.start_time) >= start && Date.parse(trace.start_time) <= end),
next_cursor: null,
};
}
const run = data.runs.find(({ trace }) => trace.summary.trace_id === parts[2]);
if (!parts[3]) return run?.trace;
if (parts[3] === "spans" && !parts[5]) return run?.details.find((span) => span.span_id === parts[4]);
if (parts[3] === "spans" && parts[5] === "error") {
const span = run?.trace.spans.find((item) => item.span_id === parts[4]);
return span
? { span_id: span.span_id, message: span.error ?? "", total_chars: span.error?.length ?? 0, next_cursor: null }
: undefined;
}
return undefined;
}
export function createLensDemo(now = Date.now()): LensDemo {
const data = createLensDemoData(now);
const read = (url: URL): unknown => {
const parts = url.pathname.split("/").filter(Boolean);
if (url.pathname === "/lens") return { lenses: data.lenses, workers: [], tracing_enabled: true };
if (url.pathname === "/lens/activity/available") return { traces: true, requests: false };
if (url.pathname === "/models" || url.pathname === "/model_group/info") return { data: [] };
if (parts[0] === "lens" && parts[2] === "runs") {
const jobs = data.lenses.find((lens) => lens.id === parts[1])?.jobs;
return parts[3]
? jobs?.find((job) => job.id === parts[3])
: jobs?.slice(Number(url.searchParams.get("offset") ?? 0));
}
return readDemoTrace(data, url);
};
return {
client: createApiClient({
getBaseUrl: () => "https://lens-demo.invalid",
fetchImpl: async (input, init) => {
if (init?.method !== "GET") return Response.json({ detail: "Demo data is read-only" }, { status: 403 });
const result = read(new URL(String(input)));
return result === undefined
? Response.json({ detail: "This item is not in the demo" }, { status: 404 })
: Response.json(result);
},
}),
copyTrace: (traceId, spanId) => {
const run = data.runs.find(({ trace }) => trace.summary.trace_id === traceId);
return JSON.stringify(spanId ? run?.details.find((span) => span.span_id === spanId) : run, null, 2);
},
};
}

54
ui/litellm-dashboard/src/app/(dashboard)/lens/page.tsx
View file

@ -1,60 +1,10 @@
"use client";
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
import { Aperture } from "lucide-react";
import { parseAsString, parseAsStringLiteral, useQueryState } from "nuqs";
import AgentTracesPage from "@/components/view_logs/TraceView/AgentTracesPage";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { isProxyAdminRole, isProxyAdminTierRole } from "@/utils/roles";
import { LensView } from "./_components/LensView";
import { LensWorkspace } from "./_components/LensWorkspace";
export default function LensPage() {
const { accessToken, userRole, isViewOnly } = useAuthorized();
const [tab, setTab] = useQueryState(
"tab",
parseAsStringLiteral(["traces", "investigations"]).withOptions({ history: "push" }),
);
const [lensId] = useQueryState("lens", parseAsString);
const defaultTab = lensId ? "investigations" : "traces";
const activeTab = tab ?? defaultTab;
if (!accessToken) return null;
return (
<main className="flex w-full min-w-0 flex-1 flex-col gap-5 p-6 md:p-8">
<h1 className="flex items-center gap-2 text-2xl font-semibold tracking-tight">
<Aperture aria-hidden="true" className="size-7" strokeWidth={1.75} />
Lens
</h1>
<Tabs
value={activeTab}
onValueChange={(value) => void setTab(value as "traces" | "investigations")}
className="min-h-0 flex-1 gap-4"
>
<TabsList variant="line" aria-label="Lens" className="w-full justify-start gap-6 border-b px-0">
<TabsTrigger value="traces" className="flex-none px-0">
Traces
</TabsTrigger>
<TabsTrigger value="investigations" className="flex-none px-0">
Investigations
</TabsTrigger>
</TabsList>
<TabsContent value="traces" keepMounted className="min-h-0">
<AgentTracesPage
accessToken={accessToken}
isActive={activeTab === "traces"}
readOnly={isViewOnly}
canMintTracingKey={isProxyAdminRole(userRole ?? "")}
/>
</TabsContent>
<TabsContent value="investigations">
{isProxyAdminTierRole(userRole ?? "") ? (
<LensView accessToken={accessToken} readOnly={isViewOnly || !isProxyAdminRole(userRole ?? "")} />
) : (
<p className="py-6 text-sm text-muted-foreground">
Investigations require proxy administrator access. You can still view your traces.
</p>
)}
</TabsContent>
</Tabs>
</main>
);
return <LensWorkspace accessToken={accessToken} userRole={userRole ?? ""} readOnly={isViewOnly} />;
}

18
ui/litellm-dashboard/src/components/view_logs/index.integration.test.tsx → ui/litellm-dashboard/src/app/(dashboard)/logs/page.integration.test.tsx
View file

@ -1,8 +1,8 @@
import { screen, waitFor } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import SpendLogsTable from "./index";
import { renderWithProviders, testQueryClient } from "../../../tests/test-utils";
import LogsPage from "./page";
import { renderWithProviders, testQueryClient } from "../../../../tests/test-utils";
const { useAuthorizedMock, useOrganizationsMock } = vi.hoisted(() => ({
useAuthorizedMock: vi.fn(),
@ -17,7 +17,7 @@ vi.mock("@/app/(dashboard)/hooks/organizations/useOrganizations", () => ({
useOrganizations: useOrganizationsMock,
}));
vi.mock("./RequestLogsPanel", () => ({
vi.mock("@/components/view_logs/RequestLogsPanel", () => ({
default: function RequestLogsPanelMock() {
return <div data-testid="request-logs-panel" />;
},
@ -47,17 +47,13 @@ const defaultProps = {
const ORG_ADMIN_MEMBERSHIPS = [{ organization_id: "org-1", members: [{ user_id: "user-1", user_role: "org_admin" }] }];
const renderAs = (sessionRole: string, organizations: unknown[] = []) => {
useAuthorizedMock.mockReturnValue({
accessToken: "sk-test",
userId: "user-1",
userRole: sessionRole,
premiumUser: true,
});
const session = { ...defaultProps, userId: defaultProps.userID, userRole: sessionRole };
useAuthorizedMock.mockReturnValue(session);
useOrganizationsMock.mockReturnValue({ data: organizations });
return renderWithProviders(<SpendLogsTable {...defaultProps} userRole={sessionRole} />);
return renderWithProviders(<LogsPage />);
};
describe("SpendLogsTable network access by role", () => {
describe("LogsPage network access by role", () => {
beforeEach(() => {
testQueryClient.clear();
vi.clearAllMocks();

28
ui/litellm-dashboard/src/components/view_logs/index.test.tsx → ui/litellm-dashboard/src/app/(dashboard)/logs/page.test.tsx
View file

@ -1,8 +1,8 @@
import { screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import SpendLogsTable from "./index";
import { renderWithProviders } from "../../../tests/test-utils";
import LogsPage from "./page";
import { renderWithProviders } from "../../../../tests/test-utils";
const { useAuthorizedMock, useOrganizationsMock } = vi.hoisted(() => ({
useAuthorizedMock: vi.fn(),
@ -17,25 +17,25 @@ vi.mock("@/app/(dashboard)/hooks/organizations/useOrganizations", () => ({
useOrganizations: useOrganizationsMock,
}));
vi.mock("./RequestLogsPanel", () => ({
vi.mock("@/components/view_logs/RequestLogsPanel", () => ({
default: function RequestLogsPanelMock({ isActive }: { isActive: boolean }) {
return <div data-testid="request-logs-panel">{isActive ? "active" : "inactive"}</div>;
},
}));
vi.mock("./AuditLogsPanel", () => ({
vi.mock("@/components/view_logs/AuditLogsPanel", () => ({
default: function AuditLogsPanelMock({ isActive }: { isActive: boolean }) {
return <div data-testid="audit-logs-panel">{isActive ? "active" : "inactive"}</div>;
},
}));
vi.mock("../DeletedKeysPage/DeletedKeysPage", () => ({
vi.mock("@/components/DeletedKeysPage/DeletedKeysPage", () => ({
default: function DeletedKeysPageMock() {
return <div data-testid="deleted-keys-page" />;
},
}));
vi.mock("../DeletedTeamsPage/DeletedTeamsPage", () => ({
vi.mock("@/components/DeletedTeamsPage/DeletedTeamsPage", () => ({
default: function DeletedTeamsPageMock() {
return <div data-testid="deleted-teams-page" />;
},
@ -52,16 +52,16 @@ const defaultProps = {
const ORG_ADMIN_MEMBERSHIPS = [{ organization_id: "org-1", members: [{ user_id: "user-1", user_role: "org_admin" }] }];
const renderAs = (sessionRole: string, organizations: unknown[] = []) => {
useAuthorizedMock.mockReturnValue({ userId: "user-1", userRole: sessionRole });
useAuthorizedMock.mockReturnValue({ ...defaultProps, userId: defaultProps.userID, userRole: sessionRole });
useOrganizationsMock.mockReturnValue({ data: organizations });
return renderWithProviders(<SpendLogsTable {...defaultProps} userRole={sessionRole} />);
return renderWithProviders(<LogsPage />);
};
const tabNames = () => screen.getAllByRole("tab").map((tab) => tab.textContent);
describe("SpendLogsTable", () => {
describe("LogsPage", () => {
beforeEach(() => {
useAuthorizedMock.mockReturnValue({ userId: "user-1", userRole: "Admin" });
useAuthorizedMock.mockReturnValue({ ...defaultProps, userId: defaultProps.userID });
useOrganizationsMock.mockReturnValue({ data: [] });
});
@ -182,17 +182,17 @@ describe("SpendLogsTable", () => {
describe("auth-not-ready guard", () => {
it("shows a loading spinner when credentials are not yet resolved", () => {
useAuthorizedMock.mockReturnValue({ userRole: "Admin" });
renderWithProviders(<SpendLogsTable {...defaultProps} accessToken={null} />);
useAuthorizedMock.mockReturnValue({ ...defaultProps, userId: defaultProps.userID, accessToken: null });
renderWithProviders(<LogsPage />);
expect(document.querySelector('[aria-busy="true"]')).toBeInTheDocument();
expect(screen.getByRole("status", { name: "Loading" })).toHaveAttribute("aria-busy", "true");
expect(screen.queryByRole("tab", { name: "Request Logs" })).not.toBeInTheDocument();
});
it("renders the tabs (no spinner) once all credentials are present", () => {
renderAs("Admin");
expect(document.querySelector('[aria-busy="true"]')).not.toBeInTheDocument();
expect(screen.queryByRole("status", { name: "Loading" })).not.toBeInTheDocument();
expect(screen.getByRole("tab", { name: "Request Logs" })).toBeInTheDocument();
});
});

75
ui/litellm-dashboard/src/app/(dashboard)/logs/page.tsx
View file

@ -1,17 +1,74 @@
"use client";
import SpendLogsTable from "@/components/view_logs";
import { useState } from "react";
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
import useCan from "@/app/(dashboard)/hooks/useCan";
import DeletedKeysPage from "@/components/DeletedKeysPage/DeletedKeysPage";
import DeletedTeamsPage from "@/components/DeletedTeamsPage/DeletedTeamsPage";
import AuditLogsPanel from "@/components/view_logs/AuditLogsPanel";
import RequestLogsPanel from "@/components/view_logs/RequestLogsPanel";
import { Page, PageTabs, PageTabsList, PageTabsTrigger, PageTabsContent } from "@/components/shared/Page";
import { UiLoadingSpinner } from "@/components/ui/ui-loading-spinner";
export default function Logs() {
type LogsTab = "request logs" | "audit logs" | "deleted keys" | "deleted teams";
export default function LogsPage() {
const { accessToken, userRole, userId, token, premiumUser } = useAuthorized();
const [activeTab, setActiveTab] = useState<LogsTab>("request logs");
const canViewAuditLogs = useCan("viewAuditLogs");
const canViewDeletedTeams = useCan("viewDeletedTeams");
const credentialsPending = !accessToken || !token;
const identityPending = !userRole || !userId;
if (credentialsPending || identityPending) {
return (
<div role="status" aria-busy="true" aria-label="Loading" className="flex h-64 items-center justify-center">
<UiLoadingSpinner className="size-8 text-primary" />
</div>
);
}
return (
<SpendLogsTable
userID={userId}
userRole={userRole}
token={token}
accessToken={accessToken}
premiumUser={premiumUser}
/>
<Page className="h-full">
<PageTabs value={activeTab} onValueChange={(value: LogsTab) => setActiveTab(value)}>
<PageTabsList>
<PageTabsTrigger value="request logs">Request Logs</PageTabsTrigger>
{canViewAuditLogs && <PageTabsTrigger value="audit logs">Audit Logs</PageTabsTrigger>}
<PageTabsTrigger value="deleted keys">Deleted Keys</PageTabsTrigger>
{canViewDeletedTeams && <PageTabsTrigger value="deleted teams">Deleted Teams</PageTabsTrigger>}
</PageTabsList>
<PageTabsContent value="request logs" keepMounted>
<RequestLogsPanel
accessToken={accessToken}
token={token}
userRole={userRole}
userID={userId}
isActive={activeTab === "request logs"}
/>
</PageTabsContent>
{canViewAuditLogs && (
<PageTabsContent value="audit logs" keepMounted>
<AuditLogsPanel
accessToken={accessToken}
token={token}
userRole={userRole}
userID={userId}
premiumUser={premiumUser ?? false}
isActive={activeTab === "audit logs"}
/>
</PageTabsContent>
)}
<PageTabsContent value="deleted keys" keepMounted>
<DeletedKeysPage />
</PageTabsContent>
{canViewDeletedTeams && (
<PageTabsContent value="deleted teams" keepMounted>
<DeletedTeamsPage />
</PageTabsContent>
)}
</PageTabs>
</Page>
);
}

21
ui/litellm-dashboard/src/app/(dashboard)/model-insights/_components/ModelInsightsView.tsx
View file

@ -1,5 +1,6 @@
"use client";
import { Page } from "@/components/shared/Page";
import React from "react";
import { Bar, BarChart, CartesianGrid, Treemap, XAxis, YAxis } from "recharts";
import { ArrowDownRight, ArrowUpRight, BarChart3, Layers, Minus } from "lucide-react";
@ -7,7 +8,7 @@ import { ArrowDownRight, ArrowUpRight, BarChart3, Layers, Minus } from "lucide-r
import { apiClient } from "@/components/networking";
import { extractErrorMessage } from "@/utils/errorUtils";
import { ProviderLogo } from "@/components/molecules/models/ProviderLogo";
import { PageHeader } from "@/components/shared/PageHeader";
import { PageHeader, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card";
import { ChartConfig, ChartContainer, ChartTooltip, ChartTooltipContent } from "@/components/ui/chart";
@ -210,12 +211,16 @@ export default function ModelInsightsView({ accessToken }: { accessToken: string
) satisfies ChartConfig;
return (
<main className="w-full space-y-6 p-8">
<PageHeader
icon={<BarChart3 />}
title="Model Leaderboard"
subtitle={`See which models your gateway used from ${data.start_date} through ${data.end_date}`}
/>
<Page>
<PageHeader>
<PageHeaderTitle>
<BarChart3 />
Model Leaderboard
</PageHeaderTitle>
<PageHeaderDescription>
See which models your gateway used from {data.start_date} through {data.end_date}
</PageHeaderDescription>
</PageHeader>
<Card aria-busy={isStale} className={isStale ? "opacity-60 transition-opacity" : "transition-opacity"}>
<CardHeader className="flex-row items-start justify-between space-y-0">
@ -375,6 +380,6 @@ export default function ModelInsightsView({ accessToken }: { accessToken: string
</p>
</CardContent>
</Card>
</main>
</Page>
);
}

81
ui/litellm-dashboard/src/app/(dashboard)/projects/_components/ProjectsPage.tsx
View file

@ -1,4 +1,5 @@
import { useCanManageProjects } from "@/app/(dashboard)/hooks/projects/projectAccess";
import { Page, PageContent } from "@/components/shared/Page";
import { useProjects } from "@/app/(dashboard)/hooks/projects/useProjects";
import { useTeams } from "@/app/(dashboard)/hooks/teams/useTeams";
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
@ -6,7 +7,7 @@ import { isUserTeamAdminForAnyTeam } from "@/utils/roles";
import { Folder, Plus, SearchIcon, X } from "lucide-react";
import { parseAsString, useQueryState } from "nuqs";
import { useMemo, useState } from "react";
import { PageHeader } from "@/components/shared/PageHeader";
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { Button } from "@/components/ui/button";
import { InputGroup, InputGroupAddon, InputGroupButton, InputGroupInput } from "@/components/ui/input-group";
import { CreateProjectModal } from "./ProjectModals/CreateProjectModal";
@ -61,51 +62,55 @@ export function ProjectsPage() {
}
return (
<div className="p-8">
<PageHeader
icon={<Folder />}
title="Projects"
subtitle="Manage projects within your teams"
primaryAction={
canCreateProject && (
<Page>
<PageHeader>
<PageHeaderTitle>
<Folder />
Projects
</PageHeaderTitle>
<PageHeaderDescription>Manage projects within your teams</PageHeaderDescription>
{canCreateProject && (
<PageHeaderControls>
<Button onClick={() => setIsCreateModalVisible(true)}>
<Plus className="size-4" />
Create Project
</Button>
)
}
/>
</PageHeaderControls>
)}
</PageHeader>
<div className="mt-6 mb-3 flex items-center">
<InputGroup className="max-w-[400px]">
<InputGroupAddon>
<SearchIcon className="size-4 text-muted-foreground" />
</InputGroupAddon>
<InputGroupInput
placeholder="Search projects by name, ID, description, or team..."
value={searchText}
onChange={(e) => setSearchText(e.target.value)}
/>
{searchText && (
<InputGroupAddon align="inline-end">
<InputGroupButton size="icon-xs" aria-label="Clear search" onClick={() => setSearchText("")}>
<X />
</InputGroupButton>
<PageContent className="gap-3">
<div className="flex items-center">
<InputGroup className="max-w-[400px]">
<InputGroupAddon>
<SearchIcon className="size-4 text-muted-foreground" />
</InputGroupAddon>
)}
</InputGroup>
</div>
<InputGroupInput
placeholder="Search projects by name, ID, description, or team..."
value={searchText}
onChange={(e) => setSearchText(e.target.value)}
/>
{searchText && (
<InputGroupAddon align="inline-end">
<InputGroupButton size="icon-xs" aria-label="Clear search" onClick={() => setSearchText("")}>
<X />
</InputGroupButton>
</InputGroupAddon>
)}
</InputGroup>
</div>
<ProjectsTable
projects={filteredProjects}
isLoading={isLoading}
isFiltered={searchText.trim().length > 0}
onProjectClick={(id) => void setSelectedProjectId(id)}
teamAliasMap={teamAliasMap}
isTeamsLoading={isTeamsLoading}
/>
<ProjectsTable
projects={filteredProjects}
isLoading={isLoading}
isFiltered={searchText.trim().length > 0}
onProjectClick={(id) => void setSelectedProjectId(id)}
teamAliasMap={teamAliasMap}
isTeamsLoading={isTeamsLoading}
/>
</PageContent>
<CreateProjectModal isOpen={isCreateModalVisible} onClose={() => setIsCreateModalVisible(false)} />
</div>
</Page>
);
}

26
ui/litellm-dashboard/src/app/(dashboard)/roi-calculator/_components/ROICalculatorView.integration.test.tsx
View file

@ -249,6 +249,32 @@ describe("ROICalculatorView", () => {
expect(screen.getAllByText("Connect GitHub to get started")).toHaveLength(1);
});
it("clearly identifies the sample report and returns to setup when exiting", async () => {
const emptySettings = { ...settings, has_github_token: false, ready: false, repos: [], estimator_model: "" };
vi.mocked(apiClient.get).mockImplementation((path: string, options) => {
if (path === "/roi-calculator/settings") return Promise.resolve(emptySettings);
if (path === "/roi-calculator/report") {
return Promise.resolve({ report: options?.query?.mode === "demo" ? { ...summary, mode: "demo" } : null });
}
return Promise.resolve(idleStatus);
});
render(<ROICalculatorView accessToken="token" />);
fireEvent.click(await screen.findByRole("button", { name: "Preview sample report" }));
expect(await screen.findByText("You’re viewing demo data")).toBeVisible();
expect(screen.getByText("Spend per estimated engineering hour")).toBeVisible();
expect(screen.queryByRole("button", { name: "Run analysis" })).not.toBeInTheDocument();
expect(screen.queryByRole("tab", { name: "Settings" })).not.toBeInTheDocument();
fireEvent.click(screen.getByRole("button", { name: "Exit demo" }));
expect(screen.getByRole("heading", { name: "Connect GitHub to get started" })).toBeVisible();
expect(screen.queryByText("You’re viewing demo data")).not.toBeInTheDocument();
expect(apiClient.post).not.toHaveBeenCalled();
expect(apiClient.put).not.toHaveBeenCalled();
});
it("returns to Overview and shows the last sync time when completion is polled from Settings", async () => {
const runningStatus = {
...idleStatus,

68
ui/litellm-dashboard/src/app/(dashboard)/roi-calculator/_components/ROICalculatorView.tsx
View file

@ -1,10 +1,12 @@
"use client";
import { Page } from "@/components/shared/Page";
import React from "react";
import { Calculator, RefreshCw } from "lucide-react";
import { apiClient } from "@/components/networking";
import { PageHeader } from "@/components/shared/PageHeader";
import { DemoNotice } from "@/components/shared/DemoNotice";
import { PageHeader, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
import { Button } from "@/components/ui/button";
import { Card, CardContent } from "@/components/ui/card";
@ -218,42 +220,34 @@ export default function ROICalculatorView({
const syncedAt = syncIsUpToDate ? summary?.synced_at : null;
return (
<main className="w-full space-y-6 p-8">
<PageHeader
icon={<Calculator />}
title="ROI Calculator"
subtitle={
<>
{summary
? `${summary.start} through ${summary.end} · UTC`
: "Compare gateway spend with estimated engineering effort for merged pull requests"}
{syncedAt && (
<span className="mt-1 block text-xs text-muted-foreground" role="status">
Last synced {formatSyncedAt(syncedAt)}
{!status.running && status.phase === "complete" && status.reused > 0
? ` · ${status.reused} of ${status.total} estimates reused`
: ""}
</span>
)}
</>
}
/>
{!liveSummary && showLiveStatus && (
<Button variant="outline" onClick={() => void previewSample()}>
Preview sample report
</Button>
)}
{sampleSummary && (
<Alert>
<AlertTitle>Sample report</AlertTitle>
<AlertDescription>
Example data only. No GitHub or model requests were made.
<Button variant="link" onClick={() => setSampleSummary(null)}>
Use my data
<Page>
<PageHeader>
<div className="flex flex-wrap items-center justify-between gap-3">
<PageHeaderTitle>
<Calculator />
ROI Calculator
</PageHeaderTitle>
{!liveSummary && showLiveStatus && (
<Button variant="outline" onClick={() => void previewSample()}>
Preview sample report
</Button>
</AlertDescription>
</Alert>
)}
)}
</div>
<PageHeaderDescription>
{summary
? `${summary.start} through ${summary.end} · UTC`
: "Compare gateway spend with estimated engineering effort for merged pull requests"}
{syncedAt && (
<span className="mt-1 block text-xs text-muted-foreground" role="status">
Last synced {formatSyncedAt(syncedAt)}
{!status.running && status.phase === "complete" && status.reused > 0
? ` · ${status.reused} of ${status.total} estimates reused`
: ""}
</span>
)}
</PageHeaderDescription>
</PageHeader>
{sampleSummary && <DemoNotice onExit={() => setSampleSummary(null)} />}
{liveSummary && showLiveStatus && (
<p className="text-xs text-muted-foreground">
{status.next_update ? `Next update ${formatSyncedAt(status.next_update)}` : scheduleLabel}
@ -372,6 +366,6 @@ export default function ROICalculatorView({
onSave={updateIdentity}
/>
)}
</main>
</Page>
);
}

7
ui/litellm-dashboard/src/components/DeletedKeysPage/DeletedKeysPage.tsx
View file

@ -1,4 +1,5 @@
"use client";
import { PageContent } from "@/components/shared/Page";
import { useState } from "react";
import { PaginationState } from "@tanstack/react-table";
import { Info } from "lucide-react";
@ -14,9 +15,9 @@ export default function DeletedKeysPage() {
const { data: keysData, isLoading } = useDeletedKeys(pagination.pageIndex + 1, pagination.pageSize);
return (
<div className="flex flex-col gap-4">
<PageContent>
{!premiumUser && (
<Alert>
<Alert className="shrink-0">
<Info />
<AlertTitle>Coming soon to Enterprise</AlertTitle>
<AlertDescription>
@ -31,6 +32,6 @@ export default function DeletedKeysPage() {
pagination={pagination}
onPaginationChange={setPagination}
/>
</div>
</PageContent>
);
}

1
ui/litellm-dashboard/src/components/DeletedKeysPage/DeletedKeysTable/DeletedKeysTable.tsx
View file

@ -44,6 +44,7 @@ export function DeletedKeysTable({
return (
<DataTable
fillHeight
data={keys}
columns={columns}
getRowId={(key, index) => key.token || String(index)}

7
ui/litellm-dashboard/src/components/DeletedTeamsPage/DeletedTeamsPage.tsx
View file

@ -1,6 +1,7 @@
"use client";
import { PaginationState } from "@tanstack/react-table";
import { Info } from "lucide-react";
import { PageContent } from "@/components/shared/Page";
import { useState } from "react";
import { Alert, AlertDescription, AlertTitle } from "@/components/shared/Alert";
import { DEFAULT_PAGE_SIZE_OPTIONS } from "@/components/shared/DataTable";
@ -17,9 +18,9 @@ export default function DeletedTeamsPage() {
const { data: teamsData, isLoading } = useDeletedTeams(pagination.pageIndex + 1, pagination.pageSize);
return (
<div className="flex flex-col gap-4">
<PageContent>
{!premiumUser && (
<Alert>
<Alert className="shrink-0">
<Info />
<AlertTitle>Coming soon to Enterprise</AlertTitle>
<AlertDescription>
@ -34,6 +35,6 @@ export default function DeletedTeamsPage() {
onPaginationChange={setPagination}
rowCount={teamsData?.total ?? 0}
/>
</div>
</PageContent>
);
}

1
ui/litellm-dashboard/src/components/DeletedTeamsPage/DeletedTeamsTable/DeletedTeamsTable.tsx
View file

@ -44,6 +44,7 @@ export function DeletedTeamsTable({
return (
<DataTable
fillHeight
data={teams}
columns={columns}
getRowId={(team, index) => team.team_id || String(index)}

10
ui/litellm-dashboard/src/components/Teams.test.tsx
View file

@ -529,15 +529,6 @@ describe("Teams - team detail deep link (?team=)", () => {
expect(onUrlUpdate.mock.calls.at(-1)![0].searchParams.has("team")).toBe(false);
await waitFor(() => expect(screen.queryByTestId("team-info-view")).not.toBeInTheDocument());
});
it("should preserve the legacy inset for the team detail view", async () => {
renderWithQueryClient(<Teams accessToken="test-token" userID="user-123" userRole="Admin" />, {
searchParams: "?team=team-from-url",
});
await waitFor(() => expect(mockTeamInfoView).toHaveBeenCalled());
expect(screen.getByRole("main")).toHaveClass("px-12", "py-6");
});
});
describe("Teams - Create Team CTA is grouped with the tabs on the left", () => {
@ -553,7 +544,6 @@ describe("Teams - Create Team CTA is grouped with the tabs on the left", () => {
const createButton = within(tabNav).getByTestId("create-team-button");
const firstTab = within(tabNav).getByRole("tab", { name: "Your Teams" });
expect(screen.getByRole("main")).toHaveClass("p-8");
expect(within(tabNav).getByRole("separator")).toBeInTheDocument();
expect(createButton.compareDocumentPosition(firstTab) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy();
});

63
ui/litellm-dashboard/src/components/Teams.tsx
View file

@ -1,3 +1,4 @@
import { Page, PageTabs, PageTabsList, PageTabsTrigger } from "@/components/shared/Page";
import { useOrganizations } from "@/app/(dashboard)/hooks/organizations/useOrganizations";
import useCan from "@/app/(dashboard)/hooks/useCan";
import AvailableTeamsPanel from "@/components/team/AvailableTeamsPanel";
@ -15,12 +16,13 @@ import { SearchSelect } from "@/components/shared/SearchSelect";
import { labelWithDocsHint, labelWithHint } from "@/components/shared/form/LabelWithHint";
import { useZodForm } from "@/lib/forms/useZodForm";
import { TagsInput } from "@/app/(dashboard)/guardrails/_components/content_filter/TagsInput";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { TabsContent } from "@/components/ui/tabs";
import { ChevronDown, Plus, Users } from "lucide-react";
import React, { useEffect, useMemo, useState } from "react";
import { z } from "zod/v4";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { PageHeader } from "@/components/shared/PageHeader";
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { ToolbarSeparator } from "@/components/shared/ToolbarSeparator";
import { Button as UIButton } from "@/components/ui/button";
import { teamsTableKeys } from "@/app/(dashboard)/hooks/teams/useTeams";
import { parseAsString, useQueryState } from "nuqs";
@ -652,7 +654,7 @@ const Teams: React.FC<TeamProps> = ({ accessToken, userID, userRole, premiumUser
];
return (
<main className={selectedTeamId ? "px-12 py-6" : "flex h-full flex-col p-8"}>
<Page className={selectedTeamId ? undefined : "h-full"}>
{selectedTeamId ? (
<TeamInfoView
teamId={selectedTeamId}
@ -672,43 +674,38 @@ const Teams: React.FC<TeamProps> = ({ accessToken, userID, userRole, premiumUser
premiumUser={premiumUser}
/>
) : (
<Tabs defaultValue={tabItems[0].key} className="min-h-0 flex-1 gap-6">
<PageHeader
icon={<Users />}
title="Teams"
subtitle="Manage teams, members, and their access to models and budgets"
primaryAction={
canCreateOrManageTeams(userRole, userID, organizations) ? (
<UIButton onClick={openCreateTeamModal} data-testid="create-team-button">
<Plus className="size-4" />
Create Team
</UIButton>
) : undefined
}
tabs={({ leadingControls }) => (
<TabsList
variant="line"
className="gap-0 p-0 [&>[data-slot=tabs-trigger]+[data-slot=tabs-trigger]]:ml-[22px]"
>
{leadingControls}
<PageTabs defaultValue={tabItems[0].key}>
<PageHeader>
<PageHeaderTitle>
<Users />
Teams
</PageHeaderTitle>
<PageHeaderDescription>Manage teams, members, and their access to models and budgets</PageHeaderDescription>
<PageHeaderControls>
<PageTabsList>
{canCreateOrManageTeams(userRole, userID, organizations) && (
<>
<UIButton onClick={openCreateTeamModal} data-testid="create-team-button">
<Plus className="size-4" />
Create Team
</UIButton>
<ToolbarSeparator className="mx-0 h-6" />
</>
)}
{tabItems.map((item) => (
<TabsTrigger
key={item.key}
value={item.key}
className="flex-none px-0 py-[7px] data-active:font-semibold"
>
<PageTabsTrigger key={item.key} value={item.key}>
{item.label}
</TabsTrigger>
</PageTabsTrigger>
))}
</TabsList>
)}
/>
</PageTabsList>
</PageHeaderControls>
</PageHeader>
{tabItems.map((item) => (
<TabsContent key={item.key} value={item.key} className={item.className}>
{item.children}
</TabsContent>
))}
</Tabs>
</PageTabs>
)}
{canCreateOrManageTeams(userRole, userID, organizations) && (
@ -1312,7 +1309,7 @@ const Teams: React.FC<TeamProps> = ({ accessToken, userID, userRole, premiumUser
</DialogContent>
</Dialog>
)}
</main>
</Page>
);
};

21
ui/litellm-dashboard/src/components/VirtualKeysPage/VirtualKeysTable.tsx
View file

@ -16,7 +16,8 @@ import {
type UrlTableStateOptions,
} from "@/components/shared/DataTable";
import { SearchSelect } from "@/components/shared/SearchSelect";
import { PageHeader } from "@/components/shared/PageHeader";
import { PageContent } from "@/components/shared/Page";
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "@/components/shared/PageHeader";
import { Input } from "@/components/ui/input";
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
import { useDebouncedValue } from "@tanstack/react-pacer/debouncer";
@ -229,13 +230,15 @@ export function VirtualKeysTable({ headerActions }: VirtualKeysTableProps) {
}
return (
<div className="flex min-h-0 flex-1 flex-col gap-6">
<PageHeader
icon={<KeyRound />}
title="Virtual Keys"
subtitle="Every key that authenticates requests to the gateway."
primaryAction={headerActions}
/>
<PageContent>
<PageHeader>
<PageHeaderTitle>
<KeyRound />
Virtual Keys
</PageHeaderTitle>
<PageHeaderDescription>Every key that authenticates requests to the gateway.</PageHeaderDescription>
{headerActions != null && <PageHeaderControls>{headerActions}</PageHeaderControls>}
</PageHeader>
<DataTable
data={keyList}
columns={columns}
@ -338,6 +341,6 @@ export function VirtualKeysTable({ headerActions }: VirtualKeysTableProps) {
</>
)}
/>
</div>
</PageContent>
);
}

12
ui/litellm-dashboard/src/components/lens/LensDemoContext.tsx Normal file
View file

@ -0,0 +1,12 @@
"use client";
import { createContext, useContext } from "react";
import type { ApiClient } from "@/lib/http/client";
export interface LensDemo {
client: ApiClient;
copyTrace: (traceId: string, spanId?: string) => string;
}
export const LensDemoContext = createContext<LensDemo | null>(null);
export const useLensDemo = () => useContext(LensDemoContext);

18
ui/litellm-dashboard/src/components/lens/LensPreviewButton.tsx Normal file
View file

@ -0,0 +1,18 @@
"use client";
import { createContext, useContext } from "react";
import { createPortal } from "react-dom";
import { Button } from "@/components/ui/button";
export const LensPreviewTarget = createContext<HTMLElement | null | undefined>(undefined);
export function LensPreviewButton({ onClick }: { onClick: () => void }) {
const target = useContext(LensPreviewTarget);
const button = (
<Button variant="outline" onClick={onClick}>
Preview sample
</Button>
);
if (target === null) return null;
return target ? createPortal(button, target) : button;
}

22
ui/litellm-dashboard/src/components/shared/DataTable/DataTable.test.tsx
View file

@ -899,28 +899,6 @@ describe("DataTable layout", () => {
expect(screen.getByTestId("data-table-head")).toHaveClass("sticky", "bg-background");
});
it("caps fillHeight at the parent's height instead of stretching to it, so a short table stays short", () => {
render(<DataTable data={CHARLIE_ALICE_BOB} columns={nameEmailColumns} fillHeight />);
const outer = screen.getByTestId("data-table-root");
const frame = screen.getByTestId("data-table-frame");
const scroller = screen.getByTestId("data-table-scroller");
// A ceiling, not a stretch: flex-1 here would hold the footer at the bottom on a two-row table.
expect(outer).toHaveClass("max-h-full", "flex-col");
expect(outer).not.toHaveClass("flex-1");
expect(frame).toHaveClass("flex-col");
expect(frame).not.toHaveClass("flex-1");
expect(scroller).not.toHaveClass("flex-1");
expect(scroller).toHaveClass("min-h-0", "overflow-auto");
expect(scroller).toHaveStyle({ maxHeight: "" });
// Without this the Table primitive's own overflow container captures the sticky header.
expect(scroller).toHaveClass("[&_[data-slot=table-container]]:overflow-visible");
// Rows pass under the header, so the semi-transparent row tint alone would let them show through.
expect(screen.getByTestId("data-table-head")).toHaveClass("sticky", "bg-background");
});
it("leaves the default layout untouched when neither height mode is set", () => {
render(<DataTable data={CHARLIE_ALICE_BOB} columns={nameEmailColumns} />);
const scroller = screen.getByTestId("data-table-scroller");

94
ui/litellm-dashboard/src/components/shared/DataTable/DataTable.tsx
View file

@ -40,7 +40,7 @@ import {
TableHeader,
TableRow,
} from "@/components/ui/table";
import { cn } from "@/lib/cva.config";
import { cn, cva } from "@/lib/cva.config";
import "./columnMeta";
import { DataTablePagination, DEFAULT_PAGE_SIZE_OPTIONS } from "./DataTablePagination";
@ -58,25 +58,40 @@ const INTERACTIVE_SELECTOR = "button, a, input, select, textarea, [role=checkbox
const noop = () => {};
/**
* Height-filling mode. The table still sizes to its rows; the parent's height is only a ceiling, so
* a short table keeps its footer under the last row and a long one scrolls its rows instead of the
* page.
*/
const FILL_CLASSES = {
outer: "flex max-h-full min-h-0 flex-col",
frame: "flex min-h-0 flex-col",
body: "min-h-0",
} as const;
const dataTableRoot = cva("w-full", {
variants: { fill: { true: "flex h-full min-h-0 flex-1 flex-col", false: null } },
});
const NO_FILL_CLASSES = { outer: "", frame: "", body: "" } as const;
const dataTableFrame = cva("overflow-hidden rounded-lg border border-border", {
variants: { fill: { true: "flex min-h-0 flex-1 flex-col", false: null } },
});
const STICKY_CLASSES = {
body: "[&_[data-slot=table-container]]:overflow-visible",
header: "bg-background",
} as const;
const dataTableScroller = cva("", {
variants: {
sticky: { true: "overflow-auto [&_[data-slot=table-container]]:overflow-visible", false: "overflow-x-auto" },
fill: { true: "min-h-0 flex-1", false: null },
stretchEmpty: { true: "[container-type:inline-size] [&_[data-slot=table-container]]:h-full", false: null },
},
});
const NO_STICKY_CLASSES = { body: "", header: "" } as const;
const dataTableTable = cva("", {
variants: {
resizable: { true: "table-fixed", false: null },
stretchEmpty: { true: "h-full", false: null },
},
});
const dataTableHeader = cva("", {
variants: { sticky: { true: "sticky top-0 z-sticky bg-background", false: null } },
});
const dataTableBody = cva("", {
variants: { stretchEmpty: { true: "h-full", false: null } },
});
const messageCell = cva("h-24 text-center align-middle text-sm whitespace-normal text-muted-foreground", {
variants: { stretch: { true: "p-0", false: null } },
});
function columnDefId<TData, TValue>(column: ColumnDef<TData, TValue>): string | undefined {
if ("id" in column && typeof column.id === "string") {
@ -314,14 +329,23 @@ function DataTableBodyRow<TData>({
);
}
function MessageRow({ colSpan, children }: { colSpan: number; children: React.ReactNode }) {
function MessageRow({
colSpan,
children,
stretch = false,
}: {
colSpan: number;
children: React.ReactNode;
stretch?: boolean;
}) {
return (
<TableRow className="hover:bg-transparent">
<TableCell
colSpan={colSpan}
className="h-24 text-center align-middle text-sm whitespace-normal text-muted-foreground"
>
{children}
<TableCell colSpan={colSpan} className={messageCell({ stretch })}>
{stretch ? (
<div className="sticky left-0 flex h-full w-[100cqw] items-center justify-center">{children}</div>
) : (
children
)}
</TableCell>
</TableRow>
);
@ -598,8 +622,7 @@ export function DataTable<TData extends RowData, TValue>(props: DataTableProps<T
const rows = table.getRowModel().rows;
const visibleColumnCount = table.getVisibleLeafColumns().length;
const stickyHeader = maxBodyHeight !== undefined || fillHeight;
const fill = fillHeight ? FILL_CLASSES : NO_FILL_CLASSES;
const sticky = stickyHeader ? STICKY_CLASSES : NO_STICKY_CLASSES;
const stretchEmpty = fillHeight && !isLoading && rows.length === 0;
const tableStyle = enableColumnResizing ? { width: table.getTotalSize(), minWidth: "100%" } : undefined;
const renderPagination = (): React.ReactNode => {
@ -636,7 +659,11 @@ export function DataTable<TData extends RowData, TValue>(props: DataTableProps<T
);
}
if (rows.length === 0) {
return <MessageRow colSpan={visibleColumnCount}>{noDataMessage ?? <DefaultEmptyState />}</MessageRow>;
return (
<MessageRow colSpan={visibleColumnCount} stretch={stretchEmpty}>
{noDataMessage ?? <DefaultEmptyState />}
</MessageRow>
);
}
return rows.map((row) => (
<DataTableBodyRow
@ -655,19 +682,16 @@ export function DataTable<TData extends RowData, TValue>(props: DataTableProps<T
const paginationNode = renderPagination();
return (
<div data-testid="data-table-root" className={cn("w-full", fill.outer)}>
<div data-testid="data-table-frame" className={cn("overflow-hidden rounded-lg border border-border", fill.frame)}>
<div data-testid="data-table-root" className={dataTableRoot({ fill: fillHeight })}>
<div data-testid="data-table-frame" className={dataTableFrame({ fill: fillHeight })}>
{toolbar !== undefined && <div className="shrink-0 border-b border-border px-4 py-3">{toolbar(table)}</div>}
<div
data-testid="data-table-scroller"
className={cn(stickyHeader ? "overflow-auto" : "overflow-x-auto", sticky.body, fill.body)}
className={dataTableScroller({ sticky: stickyHeader, fill: fillHeight, stretchEmpty })}
style={maxBodyHeight !== undefined ? { maxHeight: maxBodyHeight } : undefined}
>
<TableRoot className={enableColumnResizing ? "table-fixed" : ""} style={tableStyle}>
<TableHeader
data-testid="data-table-head"
className={cn(stickyHeader ? "sticky top-0 z-sticky" : "", sticky.header)}
>
<TableRoot className={dataTableTable({ resizable: enableColumnResizing, stretchEmpty })} style={tableStyle}>
<TableHeader data-testid="data-table-head" className={dataTableHeader({ sticky: stickyHeader })}>
{table.getHeaderGroups().map((headerGroup) => (
<TableRow key={headerGroup.id} className="bg-muted/50">
{headerGroup.headers.map((header) => (
@ -682,7 +706,7 @@ export function DataTable<TData extends RowData, TValue>(props: DataTableProps<T
</TableRow>
))}
</TableHeader>
<TableBody>{renderBody()}</TableBody>
<TableBody className={dataTableBody({ stretchEmpty })}>{renderBody()}</TableBody>
{footer !== undefined && <TableFooter>{footer(table)}</TableFooter>}
</TableRoot>
</div>

16
ui/litellm-dashboard/src/components/shared/DemoNotice.tsx Normal file
View file

@ -0,0 +1,16 @@
import { Info } from "lucide-react";
import { Button } from "@/components/ui/button";
export function DemoNotice({ onExit }: { onExit?: () => void }) {
return (
<div className="flex flex-wrap items-center justify-between gap-3 rounded-lg border border-info/20 bg-info/10 px-4 py-3">
<p role="status" className="flex items-center gap-2 text-sm font-medium text-info">
<Info aria-hidden="true" className="size-4 shrink-0" />
You’re viewing demo data
</p>
<Button variant="outline" size="sm" onClick={onExit}>
Exit demo
</Button>
</div>
);
}

40
ui/litellm-dashboard/src/components/shared/Page.tsx Normal file
View file

@ -0,0 +1,40 @@
"use client";
import type { ComponentProps } from "react";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { cn } from "@/lib/cva.config";
export function Page({ className, ...props }: ComponentProps<"div">) {
return <div className={cn("flex min-h-0 min-w-0 w-full flex-col gap-6 p-4 sm:p-6", className)} {...props} />;
}
export function PageContent({ className, ...props }: ComponentProps<"div">) {
return <div className={cn("flex min-h-0 min-w-0 flex-1 flex-col gap-6", className)} {...props} />;
}
export function PageTabs({ className, ...props }: ComponentProps<typeof Tabs>) {
return <Tabs className={cn("min-h-0 min-w-0 flex-1 gap-6", className)} {...props} />;
}
export function PageTabsList({ className, ...props }: ComponentProps<typeof TabsList>) {
return (
<TabsList
variant="line"
className={cn("h-10 w-full shrink-0 justify-start gap-6 overflow-x-auto border-b border-border p-0", className)}
{...props}
/>
);
}
export function PageTabsTrigger({ className, ...props }: ComponentProps<typeof TabsTrigger>) {
return (
<TabsTrigger
className={cn("flex-none px-0 font-normal group-data-horizontal/tabs:after:bottom-0", className)}
{...props}
/>
);
}
export function PageTabsContent({ className, ...props }: ComponentProps<typeof TabsContent>) {
return <TabsContent className={cn("flex min-h-0 min-w-0 flex-1 flex-col", className)} {...props} />;
}

99
ui/litellm-dashboard/src/components/shared/PageHeader.test.tsx
View file

@ -1,77 +1,54 @@
import { renderWithProviders, screen, within } from "@/../tests/test-utils";
import { Users } from "lucide-react";
import { describe, expect, it } from "vitest";
import { PageHeader } from "./PageHeader";
const identity = {
icon: <span>Teams icon</span>,
title: "Teams",
subtitle: "Manage teams, members, and their access to models and budgets",
};
import { PageHeader, PageHeaderControls, PageHeaderDescription, PageHeaderTitle } from "./PageHeader";
describe("PageHeader", () => {
it("should render the page identity", () => {
renderWithProviders(<PageHeader {...identity} />);
expect(screen.getByRole("heading", { name: "Teams" })).toBeInTheDocument();
expect(screen.getByText("Teams icon").parentElement).toHaveAttribute("aria-hidden", "true");
expect(screen.getByText(identity.subtitle)).toBeInTheDocument();
});
it("should apply the standard title and subtext typography", () => {
renderWithProviders(<PageHeader {...identity} />);
const icon = screen.getByText("Teams icon").parentElement;
expect(screen.getByRole("heading", { name: "Teams" })).toHaveClass("text-2xl", "font-semibold", "tracking-tight");
expect(screen.getByText(identity.subtitle)).toHaveClass("mt-1.5", "text-sm", "text-muted-foreground");
expect(icon).toHaveClass("size-5", "[&_svg]:size-5", "[&_svg]:stroke-[1.75]");
expect(icon?.parentElement).toHaveClass("gap-2.5");
});
it("should render the primary action, divider, tabs, and utilities in the standard control row", () => {
it("should name the page heading by its text alone when it carries an icon", () => {
renderWithProviders(
<PageHeader
{...identity}
primaryAction={<button>Create Team</button>}
tabs={
<div role="tablist">
<button role="tab">Your Teams</button>
</div>
}
utilities={<button>Refresh</button>}
/>,
<PageHeader>
<PageHeaderTitle>
<Users />
Teams
</PageHeaderTitle>
<PageHeaderDescription>Manage teams</PageHeaderDescription>
</PageHeader>,
);
expect(screen.getByRole("heading", { level: 1, name: "Teams" })).toBeInTheDocument();
expect(screen.getByText("Manage teams")).toBeInTheDocument();
});
it("should group the composed controls under one accessible label", () => {
renderWithProviders(
<PageHeader>
<PageHeaderTitle>Teams</PageHeaderTitle>
<PageHeaderControls>
<button>Create Team</button>
<button>Refresh</button>
</PageHeaderControls>
</PageHeader>,
);
const controls = screen.getByRole("group", { name: "Page controls" });
expect(controls).toHaveClass("mt-5", "h-9");
expect(within(controls).getByRole("separator")).toHaveClass("mx-4", "h-6");
expect(controls).toHaveTextContent("Create TeamYour TeamsRefresh");
expect(
within(controls)
.getAllByRole("button")
.map((button) => button.textContent),
).toEqual(["Create Team", "Refresh"]);
});
it("should omit the divider when tabs are absent", () => {
renderWithProviders(<PageHeader {...identity} primaryAction={<button>Create Team</button>} />);
expect(screen.queryByRole("separator")).not.toBeInTheDocument();
});
it("should provide standard controls to an embedded tab shell", () => {
it("should forward native attributes to each part", () => {
renderWithProviders(
<PageHeader
{...identity}
primaryAction={<button>Create Team</button>}
tabs={({ leadingControls, utilities }) => (
<div role="tablist">
{leadingControls}
<button role="tab">Your Teams</button>
{utilities}
</div>
)}
utilities={<button>Refresh</button>}
/>,
<PageHeader data-testid="header">
<PageHeaderTitle id="page-title">Teams</PageHeaderTitle>
<PageHeaderControls aria-label="Team controls" />
</PageHeader>,
);
const tabs = screen.getByRole("tablist");
expect(within(tabs).getByRole("separator")).toBeInTheDocument();
expect(tabs).toHaveTextContent("Create TeamYour TeamsRefresh");
expect(screen.getByTestId("header")).toContainElement(screen.getByRole("heading", { name: "Teams" }));
expect(screen.getByRole("heading", { name: "Teams" })).toHaveAttribute("id", "page-title");
expect(screen.getByRole("group", { name: "Team controls" })).toBeInTheDocument();
});
});

74
ui/litellm-dashboard/src/components/shared/PageHeader.tsx
View file

@ -1,58 +1,36 @@
"use client";
import * as React from "react";
import type { ComponentProps } from "react";
import { ToolbarSeparator } from "./ToolbarSeparator";
import { cn } from "@/lib/cva.config";
interface EmbeddedTabsSlots {
leadingControls: React.ReactNode;
utilities: React.ReactNode;
export function PageHeader({ className, ...props }: ComponentProps<"div">) {
return <div className={cn("shrink-0", className)} {...props} />;
}
interface PageHeaderProps {
title: React.ReactNode;
subtitle: React.ReactNode;
icon: React.ReactNode;
primaryAction?: React.ReactNode;
tabs?: React.ReactNode | ((slots: EmbeddedTabsSlots) => React.ReactNode);
utilities?: React.ReactNode;
}
export function PageHeader({ title, subtitle, icon, primaryAction, tabs, utilities }: PageHeaderProps) {
const leadingControls =
primaryAction == null ? null : (
<div className="flex h-9 items-center">
{primaryAction}
{tabs != null && <ToolbarSeparator className="mx-4 h-6" />}
</div>
);
const utilityControls = utilities == null ? null : <div className="flex items-center gap-2">{utilities}</div>;
const hasControlRow = primaryAction != null || tabs != null || utilities != null;
export function PageHeaderTitle({ className, ...props }: ComponentProps<"h1">) {
return (
<div>
<div className="flex items-center gap-2.5">
<span
aria-hidden="true"
className="flex size-5 flex-none items-center justify-center text-foreground [&_svg]:size-5 [&_svg]:stroke-[1.75]"
>
{icon}
</span>
<h1 className="text-2xl font-semibold tracking-tight text-foreground">{title}</h1>
</div>
<p className="mt-1.5 text-sm text-muted-foreground">{subtitle}</p>
{typeof tabs === "function" ? (
<div className="mt-5">{tabs({ leadingControls, utilities: utilityControls })}</div>
) : (
hasControlRow && (
<div className="mt-5 flex h-9 items-center" role="group" aria-label="Page controls">
{leadingControls}
{tabs}
{utilityControls != null && <div className="ml-auto">{utilityControls}</div>}
</div>
)
<h1
className={cn(
"flex items-center gap-2.5 text-2xl font-semibold tracking-tight text-foreground [&_svg]:size-5 [&_svg]:flex-none [&_svg]:stroke-[1.75]",
className,
)}
</div>
{...props}
/>
);
}
export function PageHeaderDescription({ className, ...props }: ComponentProps<"p">) {
return <p className={cn("mt-1.5 text-sm text-muted-foreground", className)} {...props} />;
}
export function PageHeaderControls({ className, ...props }: ComponentProps<"div">) {
return (
<div
role="group"
aria-label="Page controls"
className={cn("mt-5 flex min-h-9 items-center gap-2", className)}
{...props}
/>
);
}

6
ui/litellm-dashboard/src/components/view_logs/AuditLogsPanel.tsx
View file

@ -100,7 +100,7 @@ export default function AuditLogsPanel({
if (!premiumUser) {
return (
<div style={{ textAlign: "center", marginTop: "20px" }}>
<div className="flex min-h-0 flex-1 flex-col items-center justify-center overflow-y-auto text-center">
<h1 style={{ display: "block", marginBottom: "10px" }}>✨ Enterprise Feature.</h1>
<p style={{ display: "block", marginBottom: "10px" }}>
This is a LiteLLM Enterprise feature, and requires a valid key to use.
@ -128,10 +128,6 @@ export default function AuditLogsPanel({
return (
<>
<div className="flex items-center justify-between mb-4">
<h1 className="text-xl font-semibold">Audit Logs</h1>
</div>
<AuditLogsTable
data={query.data?.audit_logs ?? []}
rowCount={query.data?.total ?? 0}

1
ui/litellm-dashboard/src/components/view_logs/AuditLogsTable.tsx
View file

@ -117,6 +117,7 @@ export function AuditLogsTable({
return (
<DataTable
fillHeight
data={data}
columns={columns}
getRowId={(row) => row.id}

135
ui/litellm-dashboard/src/components/view_logs/LogsTableToolbar.tsx
View file

@ -2,72 +2,59 @@
import moment from "moment";
import { CalendarDays } from "lucide-react";
import { useState } from "react";
import { type ComponentProps, useState } from "react";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Popover, PopoverContent, PopoverTrigger } from "@/components/ui/popover";
import { Switch } from "@/components/ui/switch";
import { cn } from "@/lib/cva.config";
import { QUICK_SELECT_OPTIONS } from "./constants";
import { getTimeRangeDisplay } from "./logs_utils";
interface LogsTableToolbarProps {
const DATETIME_LOCAL_FORMAT = "YYYY-MM-DDTHH:mm";
export interface LogsTimeRange {
startTime: string;
onStartTimeChange: (value: string) => void;
endTime: string;
onEndTimeChange: (value: string) => void;
isCustomDate: boolean;
onIsCustomDateChange: (value: boolean) => void;
selectedTimeInterval: { value: number; unit: string };
onSelectedTimeIntervalChange: (value: { value: number; unit: string }) => void;
isLiveTail: boolean;
onIsLiveTailChange: (value: boolean) => void;
excludeInternalHealthChecks: boolean;
onExcludeInternalHealthChecksChange: (value: boolean) => void;
onResetToFirstPage: () => void;
onResetFilters: () => void;
interval: { value: number; unit: string };
}
export function LogsTableToolbar({
startTime,
onStartTimeChange,
endTime,
onEndTimeChange,
isCustomDate,
onIsCustomDateChange,
selectedTimeInterval,
onSelectedTimeIntervalChange,
isLiveTail,
onIsLiveTailChange,
excludeInternalHealthChecks,
onExcludeInternalHealthChecksChange,
onResetToFirstPage,
onResetFilters,
}: LogsTableToolbarProps) {
const [quickSelectOpen, setQuickSelectOpen] = useState(false);
const relativeTimeRange = (interval: LogsTimeRange["interval"]): LogsTimeRange => ({
startTime: moment()
.subtract(interval.value, interval.unit as moment.unitOfTime.DurationConstructor)
.format(DATETIME_LOCAL_FORMAT),
endTime: moment().format(DATETIME_LOCAL_FORMAT),
isCustomDate: false,
interval,
});
const applyQuickSelect = (option: { label: string; value: number; unit: string }) => {
onResetToFirstPage();
onEndTimeChange(moment().format("YYYY-MM-DDTHH:mm"));
onStartTimeChange(
moment()
.subtract(option.value, option.unit as moment.unitOfTime.DurationConstructor)
.format("YYYY-MM-DDTHH:mm"),
);
onSelectedTimeIntervalChange({ value: option.value, unit: option.unit });
onIsCustomDateChange(false);
setQuickSelectOpen(false);
};
export const defaultLogsTimeRange = (): LogsTimeRange => relativeTimeRange({ value: 24, unit: "hours" });
export function LogsToolbar({ className, ...props }: ComponentProps<"div">) {
return <div className={cn("flex flex-wrap items-center gap-2", className)} {...props} />;
}
interface LogsTimeRangePickerProps {
value: LogsTimeRange;
onValueChange: (value: LogsTimeRange) => void;
}
export function LogsTimeRangePicker({ value, onValueChange }: LogsTimeRangePickerProps) {
const [open, setOpen] = useState(false);
const selectedOption = QUICK_SELECT_OPTIONS.find(
(option) => option.value === selectedTimeInterval.value && option.unit === selectedTimeInterval.unit,
(option) => option.value === value.interval.value && option.unit === value.interval.unit,
);
const displayLabel = isCustomDate ? getTimeRangeDisplay(isCustomDate, startTime, endTime) : selectedOption?.label;
const displayLabel = value.isCustomDate
? getTimeRangeDisplay(true, value.startTime, value.endTime)
: selectedOption?.label;
return (
<div className="flex flex-wrap items-center gap-2">
<Popover open={quickSelectOpen} onOpenChange={setQuickSelectOpen}>
<>
<Popover open={open} onOpenChange={setOpen}>
<PopoverTrigger
render={
<Button variant="outline" size="sm" className="gap-2">
@ -83,7 +70,10 @@ export function LogsTableToolbar({
key={option.label}
variant="ghost"
className="w-full justify-start font-normal"
onClick={() => applyQuickSelect(option)}
onClick={() => {
onValueChange(relativeTimeRange({ value: option.value, unit: option.unit }));
setOpen(false);
}}
>
{option.label}
</Button>
@ -92,10 +82,7 @@ export function LogsTableToolbar({
<Button
variant="ghost"
className="w-full justify-start font-normal"
onClick={() => {
onIsCustomDateChange(!isCustomDate);
onResetToFirstPage();
}}
onClick={() => onValueChange({ ...value, isCustomDate: !value.isCustomDate })}
>
Custom Range
</Button>
@ -103,58 +90,32 @@ export function LogsTableToolbar({
</PopoverContent>
</Popover>
{isCustomDate && (
{value.isCustomDate && (
<div className="flex items-center gap-2">
<Input
type="datetime-local"
className="w-auto"
value={startTime}
onChange={(event) => {
onStartTimeChange(event.target.value);
onResetToFirstPage();
}}
value={value.startTime}
onChange={(event) => onValueChange({ ...value, startTime: event.target.value })}
/>
<span className="text-sm text-muted-foreground">to</span>
<Input
type="datetime-local"
className="w-auto"
value={endTime}
onChange={(event) => {
onEndTimeChange(event.target.value);
onResetToFirstPage();
}}
value={value.endTime}
onChange={(event) => onValueChange({ ...value, endTime: event.target.value })}
/>
</div>
)}
<div className="flex items-center gap-2">
<span className="text-sm font-medium">Live Tail</span>
<Switch checked={isLiveTail} onCheckedChange={onIsLiveTailChange} aria-label="Live Tail" />
</div>
<div className="flex items-center gap-2">
<span className="text-sm font-medium">Hide Health Checks</span>
<Switch
checked={excludeInternalHealthChecks}
onCheckedChange={onExcludeInternalHealthChecksChange}
aria-label="Hide Health Checks"
/>
</div>
<Button variant="outline" size="sm" onClick={onResetFilters}>
Reset Filters
</Button>
</div>
</>
);
}
export function LiveTailBanner({ onStop }: { onStop: () => void }) {
export function LogsToolbarSwitch({ label, ...props }: ComponentProps<typeof Switch> & { label: string }) {
return (
<div className="mb-4 flex items-center justify-between rounded-md border border-success/20 bg-success/10 px-4 py-2">
<span className="text-sm text-success">Auto-refreshing every 15 seconds</span>
<button type="button" onClick={onStop} className="text-sm text-success hover:text-success/80">
Stop
</button>
<div className="flex items-center gap-2">
<span className="text-sm font-medium">{label}</span>
<Switch aria-label={label} {...props} />
</div>
);
}

8
ui/litellm-dashboard/src/components/view_logs/RequestLogsPanel.test.tsx
View file

@ -792,15 +792,15 @@ describe("RequestLogsPanel", () => {
});
describe("live tail", () => {
it("shows the auto-refresh banner on the first page and hides it once stopped", async () => {
it("shows the refresh status on the first page and hides it when live tail is disabled", async () => {
const user = userEvent.setup();
renderPanel();
expect(await screen.findByText("Auto-refreshing every 15 seconds")).toBeInTheDocument();
expect(await screen.findByText("Refreshing every 15s")).toBeInTheDocument();
await user.click(screen.getByRole("button", { name: "Stop" }));
await user.click(screen.getByRole("switch", { name: "Live Tail" }));
expect(screen.queryByText("Auto-refreshing every 15 seconds")).not.toBeInTheDocument();
expect(screen.queryByText("Refreshing every 15s")).not.toBeInTheDocument();
});
});
});

68
ui/litellm-dashboard/src/components/view_logs/RequestLogsPanel.tsx
View file

@ -7,6 +7,7 @@ import moment from "moment";
import { useCallback, useEffect, useMemo, useState } from "react";
import { DEFAULT_PAGE_SIZE_OPTIONS } from "@/components/shared/DataTable";
import { Button } from "@/components/ui/button";
import { AutoRouterModelGroupsProvider } from "@/components/shared/table_cells";
import { DEBOUNCE_WAIT_MS } from "@/utils/debounceConstants";
import type { KeyResponse } from "../key_team_helpers/key_list";
@ -23,11 +24,16 @@ import {
} from "./log_filter_logic";
import { useLogDetailRouting } from "./logDetailRouting";
import { LogDetailsDrawer } from "./LogDetailsDrawer";
import { LiveTailBanner, LogsTableToolbar } from "./LogsTableToolbar";
import {
defaultLogsTimeRange,
type LogsTimeRange,
LogsTimeRangePicker,
LogsToolbar,
LogsToolbarSwitch,
} from "./LogsTableToolbar";
import { RequestLogsTable } from "./RequestLogsTable";
const PAGE_SIZE = DEFAULT_PAGE_SIZE_OPTIONS[0];
const DEFAULT_INTERVAL = { value: 24, unit: "hours" };
const matchesLogId = (log: LogEntry, logId: string) => log.request_id === logId || log.litellm_call_id === logId;
const findLogById = (logs: readonly LogEntry[], logId: string): LogEntry | null =>
logs.find((log) => log.request_id === logId) ?? logs.find((log) => log.litellm_call_id === logId) ?? null;
@ -46,10 +52,8 @@ export default function RequestLogsPanel({ accessToken, token, userRole, userID,
const [columnFilters, setColumnFilters] = useState<ColumnFiltersState>([]);
const [sessionCursors, setSessionCursors] = useState<Record<number, string>>({});
const [startTime, setStartTime] = useState<string>(moment().subtract(24, "hours").format("YYYY-MM-DDTHH:mm"));
const [endTime, setEndTime] = useState<string>(moment().format("YYYY-MM-DDTHH:mm"));
const [isCustomDate, setIsCustomDate] = useState(false);
const [selectedTimeInterval, setSelectedTimeInterval] = useState<{ value: number; unit: string }>(DEFAULT_INTERVAL);
const [timeRange, setTimeRange] = useState<LogsTimeRange>(defaultLogsTimeRange);
const { startTime, endTime, isCustomDate } = timeRange;
const [selectedKeyIdInfoView, setSelectedKeyIdInfoView] = useState<string | null>(null);
const [selectedLog, setSelectedLog] = useState<LogEntry | null>(null);
@ -232,12 +236,17 @@ export default function RequestLogsPanel({ accessToken, token, userRole, userID,
[resetToFirstPage],
);
const handleTimeRangeChange = useCallback(
(value: LogsTimeRange) => {
setTimeRange(value);
resetToFirstPage();
},
[resetToFirstPage],
);
const handleResetFilters = useCallback(() => {
setColumnFilters([]);
setStartTime(moment().subtract(24, "hours").format("YYYY-MM-DDTHH:mm"));
setEndTime(moment().format("YYYY-MM-DDTHH:mm"));
setIsCustomDate(false);
setSelectedTimeInterval(DEFAULT_INTERVAL);
setTimeRange(defaultLogsTimeRange());
resetToFirstPage();
}, [resetToFirstPage]);
@ -288,12 +297,6 @@ export default function RequestLogsPanel({ accessToken, token, userRole, userID,
return (
<AutoRouterModelGroupsProvider>
<div className="flex items-center justify-between mb-4">
<h1 className="text-xl font-semibold">Request Logs</h1>
</div>
{isLiveTail && pagination.pageIndex === 0 && <LiveTailBanner onStop={() => setIsLiveTail(false)} />}
<RequestLogsTable
data={rows}
rowCount={rowCount}
@ -314,22 +317,23 @@ export default function RequestLogsPanel({ accessToken, token, userRole, userID,
teams={allTeams ?? []}
logsWindow={logsWindow}
toolbarChildren={
<LogsTableToolbar
startTime={startTime}
onStartTimeChange={setStartTime}
endTime={endTime}
onEndTimeChange={setEndTime}
isCustomDate={isCustomDate}
onIsCustomDateChange={setIsCustomDate}
selectedTimeInterval={selectedTimeInterval}
onSelectedTimeIntervalChange={setSelectedTimeInterval}
isLiveTail={isLiveTail}
onIsLiveTailChange={setIsLiveTail}
excludeInternalHealthChecks={excludeInternalHealthChecks}
onExcludeInternalHealthChecksChange={handleExcludeInternalHealthChecksChange}
onResetToFirstPage={resetToFirstPage}
onResetFilters={handleResetFilters}
/>
<LogsToolbar>
<LogsTimeRangePicker value={timeRange} onValueChange={handleTimeRangeChange} />
<LogsToolbarSwitch label="Live Tail" checked={isLiveTail} onCheckedChange={setIsLiveTail} />
{isLiveTail && pagination.pageIndex === 0 && (
<span role="status" className="whitespace-nowrap text-xs text-muted-foreground">
Refreshing every 15s
</span>
)}
<LogsToolbarSwitch
label="Hide Health Checks"
checked={excludeInternalHealthChecks}
onCheckedChange={handleExcludeInternalHealthChecksChange}
/>
<Button variant="outline" size="sm" onClick={handleResetFilters}>
Reset Filters
</Button>
</LogsToolbar>
}
/>

9
ui/litellm-dashboard/src/components/view_logs/TraceView/AgentTracesPage.tsx
View file

@ -1,4 +1,5 @@
"use client";
import { useLensDemo } from "@/components/lens/LensDemoContext";
import moment from "moment";
import { useMemo, useState } from "react";
@ -13,12 +14,15 @@ export default function AgentTracesPage({
isActive = true,
readOnly = false,
canMintTracingKey = false,
onDemo,
}: {
accessToken: string;
isActive?: boolean;
readOnly?: boolean;
canMintTracingKey?: boolean;
onDemo?: () => void;
}) {
const demo = useLensDemo();
const [rangeHours, setRangeHours] = useState(DEFAULT_RANGE_HOURS);
const [live, setLive] = useState(true);
const [anchor, setAnchor] = useState(() => moment());
@ -36,16 +40,17 @@ export default function AgentTracesPage({
};
return (
<div className="flex min-h-0 flex-1 flex-col pt-3">
<div className="flex min-h-0 flex-1 flex-col">
<AgentTracesSection
accessToken={accessToken}
isActive={isActive}
startTime={startTime}
endTime={endTime}
isCustomDate={false}
isLiveTail={live}
isLiveTail={live && !demo}
readOnly={readOnly}
canMintTracingKey={canMintTracingKey}
onDemo={onDemo}
timeControls={{ rangeHours, onRangeHoursChange: changeRange, onLiveChange: setLive }}
/>
</div>

509
ui/litellm-dashboard/src/components/view_logs/TraceView/AgentTracesSection.integration.test.tsx Normal file
View file

@ -0,0 +1,509 @@
import { act, fireEvent, screen, waitFor, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { ApiError } from "@/lib/http/client";
import { chooseSelectOption, renderWithProviders, testQueryClient } from "../../../../tests/test-utils";
import traceList from "./__fixtures__/trace_list.json";
import AgentTracesPage from "./AgentTracesPage";
import { AgentTracesSection, type TimeControls } from "./AgentTracesSection";
import type { TracePage, TraceSummary } from "./traceTypes";
vi.mock("../../networking", () => ({
apiClient: { get: vi.fn(), post: vi.fn() },
agentTraceListCall: vi.fn(),
sendOtlpTraceCall: vi.fn(),
agentTraceCall: vi.fn(),
agentTraceSpanCall: vi.fn(),
getProxyBaseUrl: () => "http://localhost:4000",
}));
vi.mock("./TraceDrawer", () => ({
RunView: ({ traceId, onBack }: { traceId: string; onBack: () => void }) => (
<div data-testid="run-view">
run {traceId}
<button onClick={onBack}>back</button>
</div>
),
}));
import { agentTraceListCall, apiClient } from "../../networking";
const runs = (traceList as TracePage).data as TraceSummary[];
const renderSection = () =>
renderWithProviders(
<AgentTracesSection
accessToken="sk-test"
isActive
startTime="2026-09-29T00:00"
endTime="2026-09-30T00:00"
isCustomDate={false}
isLiveTail={false}
onDemo={vi.fn()}
/>,
);
// A UTC-pinned day around the fixture runs (2026-09-30 ~06:43 UTC), so they land in the same bucket in any timezone.
const renderWindowed = (timeControls?: TimeControls) =>
renderWithProviders(
<AgentTracesSection
accessToken="sk-test"
isActive
startTime="2026-09-30T00:00Z"
endTime="2026-10-01T00:00Z"
isCustomDate
isLiveTail={false}
timeControls={timeControls}
/>,
);
const bucketRunCounts = () =>
screen.getAllByTestId("timeline-bucket").map((bucket) => Number(bucket.getAttribute("data-runs")));
describe("AgentTracesSection", () => {
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});
beforeEach(() => {
vi.spyOn(HTMLElement.prototype, "getBoundingClientRect").mockReturnValue({
left: 0,
width: 600,
top: 0,
height: 56,
right: 600,
bottom: 56,
x: 0,
y: 0,
toJSON: () => ({}),
} as DOMRect);
testQueryClient.clear();
vi.mocked(agentTraceListCall).mockReset();
vi.mocked(apiClient.get).mockResolvedValue({ data: [] });
});
it.each([
[401, "Your session is no longer valid. Sign out and sign in again."],
[403, "Your account does not have access to these traces."],
])("stops live polling after HTTP %s and explains how to recover", async (status, message) => {
vi.useFakeTimers();
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Private token details", Number(status), {}));
renderWithProviders(
<AgentTracesSection
accessToken="sk-test"
isActive
startTime="2026-09-29T00:00"
endTime="2026-09-30T00:00"
isCustomDate={false}
isLiveTail
/>,
);
await act(async () => {
await vi.advanceTimersByTimeAsync(60_000);
});
expect(screen.getByText(`Could not load runs: ${message}`)).toBeVisible();
expect(screen.queryByText(/Private token details/)).not.toBeInTheDocument();
expect(agentTraceListCall).toHaveBeenCalledOnce();
expect(screen.getByTestId("runs-footer")).toHaveTextContent("Update failed");
});
it("renders the setup snippet when the proxy answers 501", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(
new ApiError("Agent tracing is not enabled", 501, { detail: "Agent tracing is not enabled" }),
);
renderSection();
const card = await screen.findByTestId("tracing-setup-card");
expect(card).toHaveTextContent("Tracing is not enabled");
expect(screen.getByRole("button", { name: "Preview sample" })).toBeVisible();
expect(card).toHaveTextContent("type: clickhouse");
expect(card).toHaveTextContent("url: os.environ/CLICKHOUSE_URL");
expect(screen.getByRole("button", { name: "Check setup" })).toBeEnabled();
expect(card).not.toHaveTextContent(/langsmith/i);
expect(card).toHaveTextContent("ClickHouse and proxy setup");
expect(card).toHaveTextContent("Ask your proxy administrator");
});
it("shows the waiting guide when tracing is on but no runs have arrived", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
renderSection();
const card = await screen.findByTestId("tracing-setup-card");
expect(card).toHaveTextContent("Connect your agent");
expect(card).toHaveTextContent("Waiting for your first trace");
expect(screen.getByRole("button", { name: "Preview sample" })).toBeVisible();
expect(card).not.toHaveTextContent("store: clickhouse");
});
it("keeps the trace list available when traces exist outside the current time window", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
vi.mocked(apiClient.get).mockResolvedValue(traceList);
renderSection();
expect(await screen.findByText("No runs match these filters.")).toBeVisible();
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
expect(apiClient.get).toHaveBeenCalledWith("/v1/traces", { accessToken: "sk-test", query: { start_ms: 0 } });
});
it("checks proxy readiness, waits for an agent, and confirms receipt using actual query results", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Tracing is not enabled", 501, {}));
renderSection();
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
fireEvent.click(checkSetup);
expect(await screen.findByRole("heading", { name: "Connect your agent" })).toBeVisible();
expect(screen.getByText("Waiting for your first trace")).toBeVisible();
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
fireEvent.click(screen.getByRole("button", { name: "Check for traces" }));
expect(await screen.findAllByTestId("agent-trace-row")).toHaveLength(runs.length);
expect(screen.getByText("Traces received. Select a run to inspect it.")).toBeVisible();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
});
it("keeps setup visible while checking and explains when tracing is still disabled", async () => {
const failure = new ApiError("Tracing is not enabled", 501, {});
vi.mocked(agentTraceListCall).mockRejectedValue(failure);
renderSection();
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
let rejectCheck: (error: Error) => void = () => {};
vi.mocked(agentTraceListCall).mockImplementationOnce(
() =>
new Promise((_, reject) => {
rejectCheck = reject;
}),
);
fireEvent.click(checkSetup);
expect(await screen.findByRole("button", { name: "Checking…" })).toBeDisabled();
expect(screen.getByRole("heading", { name: "Enable tracing" })).toBeVisible();
await act(async () => rejectCheck(failure));
expect(await screen.findByText(/Tracing is still unavailable/)).toBeVisible();
expect(screen.getByRole("button", { name: "Check setup" })).toBeEnabled();
});
it("keeps received traces and the open drawer visible during subsequent fetches", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Tracing is not enabled", 501, {}));
renderSection();
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
fireEvent.click(checkSetup);
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
const drawer = screen.getByRole("complementary", { name: "Trace details" });
let finishRefresh: (page: TracePage) => void = () => {};
vi.mocked(agentTraceListCall).mockImplementationOnce(
() =>
new Promise((resolve) => {
finishRefresh = resolve;
}),
);
await act(async () => {
void testQueryClient.invalidateQueries({ queryKey: ["agentTraces"] });
});
expect(await screen.findByText("Updating…")).toBeVisible();
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(runs.length);
expect(screen.getByRole("complementary", { name: "Trace details" })).toBe(drawer);
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
await act(async () => finishRefresh(traceList as TracePage));
});
it("separates a failed history check from the empty list and retries that check", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
vi.mocked(apiClient.get).mockRejectedValue(new ApiError("History unavailable", 503, {}));
renderSection();
expect(await screen.findByRole("alert")).toHaveTextContent("Could not check earlier traces. History unavailable");
expect(screen.getByText("No runs match these filters.")).toBeVisible();
expect(screen.queryByText(/Could not load runs/)).not.toBeInTheDocument();
vi.mocked(apiClient.get).mockResolvedValue(traceList);
fireEvent.click(screen.getByRole("button", { name: "Retry trace check" }));
await waitFor(() => expect(screen.queryByRole("alert")).not.toBeInTheDocument());
expect(screen.getByText("No runs match these filters.")).toBeVisible();
expect(agentTraceListCall).toHaveBeenCalledTimes(1);
});
it("treats a proxy without the trace routes (404) like tracing being off", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Not Found", 404, { detail: "Not Found" }));
renderSection();
const card = await screen.findByTestId("tracing-setup-card");
expect(card).toHaveTextContent("Tracing is not enabled");
expect(card).toHaveTextContent("url: os.environ/CLICKHOUSE_URL");
});
it("lists every run with its input, counts and failed column", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
expect(rows).toHaveLength(runs.length);
const lead = rows.find((row) => row.textContent?.includes("Should we store OTEL agent spans"));
expect(lead).toBeDefined();
const failed = rows.find((row) => row.textContent?.includes("acme-404")) as HTMLElement;
expect(within(failed).getByLabelText("2 errors")).toBeInTheDocument();
expect(screen.getByText(`${runs.length} runs`)).toBeInTheDocument();
expect(screen.getByRole("columnheader", { name: "Cost" })).toBeInTheDocument();
expect(within(failed).getByText("—")).toBeInTheDocument();
});
it("shows the spend returned for a run", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({
...(traceList as TracePage),
data: [{ ...runs[0], spend: 0.025 }],
});
renderSection();
const row = await screen.findByTestId("agent-trace-row");
expect(within(row).getByText("$0.03")).toBeInTheDocument();
});
it("filters by input text and by trace id", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
await screen.findAllByTestId("agent-trace-row");
const search = screen.getByLabelText("Search runs");
fireEvent.change(search, { target: { value: "acme-404" } });
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(1);
const lead = runs.find((r) => r.name === "research_lead") as TraceSummary;
fireEvent.change(search, { target: { value: lead.trace_id.slice(0, 10) } });
const rows = screen.getAllByTestId("agent-trace-row");
expect(rows).toHaveLength(1);
expect(rows[0]).toHaveTextContent("Should we store OTEL agent spans");
});
it("uses recorded agent names for the column and filter even when services are shared", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({
...(traceList as TracePage),
data: [
...runs.slice(1).map((run) => ({ ...run, service: "shared-app", agent_names: ["research-agent"] })),
{ ...runs[0], service: "shared-app", agent_names: ["billing-agent", "review-agent"] },
],
});
const user = userEvent.setup();
renderSection();
await screen.findAllByTestId("agent-trace-row");
expect(screen.getByRole("columnheader", { name: "Agent" })).toBeInTheDocument();
const agentFilter = screen.getByRole("combobox", { name: "Filter by agent" });
expect(agentFilter).toHaveTextContent("All agents");
await chooseSelectOption(user, agentFilter, "billing-agent");
const rows = screen.getAllByTestId("agent-trace-row");
expect(rows).toHaveLength(1);
expect(rows[0]).toHaveTextContent("billing-agent");
expect(rows[0]).not.toHaveTextContent("shared-app");
await chooseSelectOption(user, agentFilter, "review-agent");
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(1);
await chooseSelectOption(user, agentFilter, "All agents");
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(runs.length);
});
it("shows each run's agent name with the logo of the SDK that produced it", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({
...(traceList as TracePage),
data: [
{ ...runs[0], agent_names: ["research-bot"], frameworks: ["claude-agent-sdk", "claude-code"] },
{ ...runs[1], agent_names: [], frameworks: ["claude-code"] },
{ ...runs[2], frameworks: [] },
],
});
renderSection();
const [sdkRun, cliRun, plainRun] = await screen.findAllByTestId("agent-trace-row");
const agentCell = (row: HTMLElement) => within(row).getAllByRole("cell")[1];
expect(agentCell(sdkRun)).toHaveTextContent(/^research-bot$/);
expect(agentCell(sdkRun)).toHaveAttribute("title", "research-bot · Claude Agent SDK");
expect(within(sdkRun).getByRole("img", { name: "Claude Agent SDK logo", hidden: true })).toHaveAttribute(
"src",
expect.stringContaining("anthropic.svg"),
);
expect(agentCell(cliRun)).toHaveTextContent(/^Claude Code$/);
expect(within(plainRun).queryByRole("img", { hidden: true })).not.toBeInTheDocument();
expect(agentCell(plainRun)).toHaveTextContent((runs[2].agent_names ?? [runs[2].service]).join(", "));
});
it("opens a run in a side drawer over the list and swaps runs without closing it", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
const drawer = screen.getByRole("complementary", { name: "Trace details" });
expect(within(drawer).getByTestId("run-view")).toHaveTextContent(`run ${runs[0].trace_id}`);
expect(screen.getByTestId("runs-table")).toBeInTheDocument();
expect(rows[0]).toHaveAttribute("aria-selected", "true");
fireEvent.click(rows[1]);
expect(screen.getByRole("complementary", { name: "Trace details" })).toBe(drawer);
expect(within(drawer).getByTestId("run-view")).toHaveTextContent(`run ${runs[1].trace_id}`);
expect(rows[1]).toHaveAttribute("aria-selected", "true");
expect(rows[0]).toHaveAttribute("aria-selected", "false");
});
it("closes the drawer when the open row is clicked again or Escape is pressed", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
fireEvent.click(rows[0]);
expect(rows[0]).toHaveAttribute("aria-selected", "false");
fireEvent.click(rows[1]);
fireEvent.keyDown(window, { key: "Escape" });
expect(rows[1]).toHaveAttribute("aria-selected", "false");
});
it("moves to the next and previous run with j / k and the header arrows", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
fireEvent.keyDown(window, { key: "j" });
expect(screen.getByTestId("run-view")).toHaveTextContent(`run ${runs[1].trace_id}`);
fireEvent.keyDown(window, { key: "k" });
expect(screen.getByTestId("run-view")).toHaveTextContent(`run ${runs[0].trace_id}`);
expect(screen.getByRole("button", { name: "Previous trace (K)" })).toBeDisabled();
fireEvent.click(screen.getByRole("button", { name: "Next trace (J)" }));
expect(screen.getByTestId("run-view")).toHaveTextContent(`run ${runs[1].trace_id}`);
});
it("plots every loaded run on the timeline", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWindowed();
await screen.findAllByTestId("agent-trace-row");
expect(screen.getByTestId("traces-timeline")).toBeInTheDocument();
const counts = bucketRunCounts();
expect(counts).toHaveLength(60);
expect(counts.reduce((a, b) => a + b, 0)).toBe(runs.length);
});
it("zooms by dragging, resizes and pans the bracket, and clears with Esc", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWindowed();
await screen.findAllByTestId("agent-trace-row");
const area = screen.getByTestId("timeline-area");
const x = (bucket: number) => bucket * 10 + 5;
const drag = (target: HTMLElement, from: number, to: number) => {
fireEvent.pointerDown(target, { clientX: x(from), pointerId: 1 });
fireEvent.pointerMove(area, { clientX: x(to), pointerId: 1 });
fireEvent.pointerUp(area, { clientX: x(to), pointerId: 1 });
};
const rowCount = () => screen.queryAllByTestId("agent-trace-row").length;
const withRuns = bucketRunCounts().flatMap((count, i) => (count > 0 ? [i] : []));
const first = withRuns[0];
// The pan below moves a [0, first] bracket to the far right; it must end up clear of every run.
expect(first).toBeGreaterThan(1);
expect(first).toBeLessThan(30);
drag(area, 0, 1);
expect(screen.getByTestId("timeline-selection")).toBeInTheDocument();
expect(rowCount()).toBe(0);
drag(screen.getByTestId("timeline-handle-hi"), 1, first);
expect(rowCount()).toBeGreaterThan(0);
drag(screen.getByTestId("timeline-selection"), 1, 1 - first);
expect(rowCount()).toBeGreaterThan(0);
drag(screen.getByTestId("timeline-selection"), 0, 59);
expect(rowCount()).toBe(0);
fireEvent.keyDown(screen.getByTestId("traces-timeline"), { key: "Escape" });
expect(screen.queryByTestId("timeline-selection")).not.toBeInTheDocument();
expect(rowCount()).toBe(runs.length);
});
it("clears timeline zoom when refreshed", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWindowed({ rangeHours: 24, onRangeHoursChange: () => {}, onLiveChange: () => {} });
await screen.findAllByTestId("agent-trace-row");
const area = screen.getByTestId("timeline-area");
const x = (bucket: number) => bucket * 10 + 5;
fireEvent.pointerDown(area, { clientX: x(0), pointerId: 1 });
fireEvent.pointerMove(area, { clientX: x(1), pointerId: 1 });
fireEvent.pointerUp(area, { clientX: x(1), pointerId: 1 });
expect(screen.getByTestId("timeline-selection")).toBeInTheDocument();
expect(screen.queryAllByTestId("agent-trace-row")).toHaveLength(0);
fireEvent.click(screen.getByRole("button", { name: "Refresh" }));
expect(screen.queryByTestId("timeline-selection")).not.toBeInTheDocument();
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(runs.length);
});
});
describe("AgentTracesPage", () => {
beforeEach(() => {
testQueryClient.clear();
vi.mocked(agentTraceListCall).mockReset();
vi.mocked(apiClient.get).mockResolvedValue({ data: [] });
});
it("shows the actual range, switches presets from the popover, and toggles Live", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWithProviders(<AgentTracesPage accessToken="sk-test" />);
await screen.findByTestId("runs-table");
const trigger = screen.getByRole("button", { name: "Time range" });
expect(trigger).toHaveTextContent(/ to /);
expect(screen.getByTestId("traces-timeline")).toHaveTextContent("Total 1d");
fireEvent.click(trigger);
fireEvent.click(await screen.findByRole("menuitemradio", { name: "Last 7 days" }));
expect(await screen.findByText("Total 7d")).toBeInTheDocument();
const last = vi.mocked(agentTraceListCall).mock.calls.at(-1)?.[0];
expect((last?.endMs ?? 0) - (last?.startMs ?? 0)).toBeGreaterThanOrEqual(7 * 24 * 3600 * 1000 - 60_000);
const live = screen.getByRole("button", { name: "Live" });
expect(live).toHaveAttribute("aria-pressed", "true");
fireEvent.click(live);
expect(live).toHaveAttribute("aria-pressed", "false");
expect(screen.getByRole("button", { name: "Refresh" })).toBeEnabled();
});
it("refreshes the trace list", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWithProviders(<AgentTracesPage accessToken="sk-test" />);
await screen.findByTestId("runs-table");
const callsBeforeRefresh = vi.mocked(agentTraceListCall).mock.calls.length;
fireEvent.click(screen.getByRole("button", { name: "Refresh" }));
await waitFor(() => expect(vi.mocked(agentTraceListCall).mock.calls.length).toBeGreaterThan(callsBeforeRefresh));
});
it("keeps the time controls on an empty range the user picked, instead of showing onboarding", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWithProviders(<AgentTracesPage accessToken="sk-test" />);
await screen.findByTestId("runs-table");
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
fireEvent.click(screen.getByRole("button", { name: "Time range" }));
fireEvent.click(await screen.findByRole("menuitemradio", { name: "Last hour" }));
expect(await screen.findByText("No runs match these filters.")).toBeInTheDocument();
expect(screen.getByRole("button", { name: "Time range" })).toBeInTheDocument();
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
});
it("asks the proxy for the last 24 hours by default", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWithProviders(<AgentTracesPage accessToken="sk-test" />);
await screen.findByTestId("runs-table");
const { startMs, endMs } = vi.mocked(agentTraceListCall).mock.calls[0][0];
expect(endMs - startMs).toBeGreaterThanOrEqual(24 * 3600 * 1000 - 60_000);
expect(endMs - startMs).toBeLessThan(24 * 3600 * 1000 + 120_000);
});
});

471
ui/litellm-dashboard/src/components/view_logs/TraceView/AgentTracesSection.test.tsx
View file

@ -1,329 +1,11 @@
import { act, fireEvent, screen, waitFor, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { ApiError } from "@/lib/http/client";
import { chooseSelectOption, renderWithProviders, testQueryClient } from "../../../../tests/test-utils";
import { describe, expect, it } from "vitest";
import { filterRuns } from "./AgentTracesSection";
import traceList from "./__fixtures__/trace_list.json";
import AgentTracesPage from "./AgentTracesPage";
import { AgentTracesSection, filterRuns } from "./AgentTracesSection";
import type { TracePage, TraceSummary } from "./traceTypes";
vi.mock("../../networking", () => ({
apiClient: { get: vi.fn(), post: vi.fn() },
agentTraceListCall: vi.fn(),
sendOtlpTraceCall: vi.fn(),
agentTraceCall: vi.fn(),
agentTraceSpanCall: vi.fn(),
getProxyBaseUrl: () => "http://localhost:4000",
}));
vi.mock("./TraceDrawer", () => ({
RunView: ({ traceId, onBack }: { traceId: string; onBack: () => void }) => (
<div data-testid="run-view">
run {traceId}
<button onClick={onBack}>back</button>
</div>
),
}));
import { agentTraceListCall, apiClient } from "../../networking";
const runs = (traceList as TracePage).data as TraceSummary[];
const renderSection = () =>
renderWithProviders(
<AgentTracesSection
accessToken="sk-test"
isActive
startTime="2026-09-29T00:00"
endTime="2026-09-30T00:00"
isCustomDate={false}
isLiveTail={false}
/>,
);
// A UTC-pinned day around the fixture runs (2026-09-30 ~06:43 UTC), so they land in the same bucket in any timezone.
const renderWindowed = () =>
renderWithProviders(
<AgentTracesSection
accessToken="sk-test"
isActive
startTime="2026-09-30T00:00Z"
endTime="2026-10-01T00:00Z"
isCustomDate
isLiveTail={false}
/>,
);
const bucketRunCounts = () =>
screen.getAllByTestId("timeline-bucket").map((bucket) => Number(bucket.getAttribute("data-runs")));
describe("AgentTracesSection", () => {
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});
beforeEach(() => {
vi.spyOn(HTMLElement.prototype, "getBoundingClientRect").mockReturnValue({
left: 0,
width: 600,
top: 0,
height: 56,
right: 600,
bottom: 56,
x: 0,
y: 0,
toJSON: () => ({}),
} as DOMRect);
testQueryClient.clear();
vi.mocked(agentTraceListCall).mockReset();
vi.mocked(apiClient.get).mockResolvedValue({ data: [] });
});
it.each([
[401, "Your session is no longer valid. Sign out and sign in again."],
[403, "Your account does not have access to these traces."],
])("stops live polling after HTTP %s and explains how to recover", async (status, message) => {
vi.useFakeTimers();
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Private token details", Number(status), {}));
renderWithProviders(
<AgentTracesSection
accessToken="sk-test"
isActive
startTime="2026-09-29T00:00"
endTime="2026-09-30T00:00"
isCustomDate={false}
isLiveTail
/>,
);
await act(async () => {
await vi.advanceTimersByTimeAsync(60_000);
});
expect(screen.getByText(`Could not load runs: ${message}`)).toBeVisible();
expect(screen.queryByText(/Private token details/)).not.toBeInTheDocument();
expect(agentTraceListCall).toHaveBeenCalledOnce();
expect(screen.getByTestId("runs-footer")).toHaveTextContent("Update failed");
});
it("renders the setup snippet when the proxy answers 501", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(
new ApiError("Agent tracing is not enabled", 501, { detail: "Agent tracing is not enabled" }),
);
renderSection();
const card = await screen.findByTestId("tracing-setup-card");
expect(card).toHaveTextContent("Tracing is not enabled");
expect(card).toHaveTextContent("type: clickhouse");
expect(card).toHaveTextContent("url: os.environ/CLICKHOUSE_URL");
expect(screen.getByRole("button", { name: "Check setup" })).toBeEnabled();
expect(card).not.toHaveTextContent(/langsmith/i);
expect(card).toHaveTextContent("ClickHouse and proxy setup");
expect(card).toHaveTextContent("Ask your proxy administrator");
});
it("shows the waiting guide when tracing is on but no runs have arrived", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
renderSection();
const card = await screen.findByTestId("tracing-setup-card");
expect(card).toHaveTextContent("Connect your agent");
expect(card).toHaveTextContent("Waiting for your first trace");
expect(card).not.toHaveTextContent("store: clickhouse");
});
it("keeps the trace list available when traces exist outside the current time window", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
vi.mocked(apiClient.get).mockResolvedValue(traceList);
renderSection();
expect(await screen.findByText("No runs match these filters.")).toBeVisible();
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
expect(apiClient.get).toHaveBeenCalledWith("/v1/traces", { accessToken: "sk-test", query: { start_ms: 0 } });
});
it("checks proxy readiness, waits for an agent, and confirms receipt using actual query results", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Tracing is not enabled", 501, {}));
renderSection();
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
fireEvent.click(checkSetup);
expect(await screen.findByRole("heading", { name: "Connect your agent" })).toBeVisible();
expect(screen.getByText("Waiting for your first trace")).toBeVisible();
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
fireEvent.click(screen.getByRole("button", { name: "Check for traces" }));
expect(await screen.findAllByTestId("agent-trace-row")).toHaveLength(runs.length);
expect(screen.getByText("Traces received. Select a run to inspect it.")).toBeVisible();
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
});
it("keeps setup visible while checking and explains when tracing is still disabled", async () => {
const failure = new ApiError("Tracing is not enabled", 501, {});
vi.mocked(agentTraceListCall).mockRejectedValue(failure);
renderSection();
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
let rejectCheck: (error: Error) => void = () => {};
vi.mocked(agentTraceListCall).mockImplementationOnce(
() =>
new Promise((_, reject) => {
rejectCheck = reject;
}),
);
fireEvent.click(checkSetup);
expect(await screen.findByRole("button", { name: "Checking…" })).toBeDisabled();
expect(screen.getByRole("heading", { name: "Enable tracing" })).toBeVisible();
await act(async () => rejectCheck(failure));
expect(await screen.findByText(/Tracing is still unavailable/)).toBeVisible();
expect(screen.getByRole("button", { name: "Check setup" })).toBeEnabled();
});
it("keeps received traces and the open drawer visible during subsequent fetches", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Tracing is not enabled", 501, {}));
renderSection();
const checkSetup = await screen.findByRole("button", { name: "Check setup" });
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
fireEvent.click(checkSetup);
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
const drawer = screen.getByRole("complementary", { name: "Trace details" });
let finishRefresh: (page: TracePage) => void = () => {};
vi.mocked(agentTraceListCall).mockImplementationOnce(
() =>
new Promise((resolve) => {
finishRefresh = resolve;
}),
);
await act(async () => {
void testQueryClient.invalidateQueries({ queryKey: ["agentTraces"] });
});
expect(await screen.findByText("Updating…")).toBeVisible();
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(runs.length);
expect(screen.getByRole("complementary", { name: "Trace details" })).toBe(drawer);
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
await act(async () => finishRefresh(traceList as TracePage));
});
it("separates a failed history check from the empty list and retries that check", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
vi.mocked(apiClient.get).mockRejectedValue(new ApiError("History unavailable", 503, {}));
renderSection();
expect(await screen.findByRole("alert")).toHaveTextContent("Could not check earlier traces. History unavailable");
expect(screen.getByText("No runs match these filters.")).toBeVisible();
expect(screen.queryByText(/Could not load runs/)).not.toBeInTheDocument();
vi.mocked(apiClient.get).mockResolvedValue(traceList);
fireEvent.click(screen.getByRole("button", { name: "Retry trace check" }));
await waitFor(() => expect(screen.queryByRole("alert")).not.toBeInTheDocument());
expect(screen.getByText("No runs match these filters.")).toBeVisible();
expect(agentTraceListCall).toHaveBeenCalledTimes(1);
});
it("treats a proxy without the trace routes (404) like tracing being off", async () => {
vi.mocked(agentTraceListCall).mockRejectedValue(new ApiError("Not Found", 404, { detail: "Not Found" }));
renderSection();
const card = await screen.findByTestId("tracing-setup-card");
expect(card).toHaveTextContent("Tracing is not enabled");
expect(card).toHaveTextContent("url: os.environ/CLICKHOUSE_URL");
});
it("lists every run with its input, counts and failed column", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
expect(rows).toHaveLength(runs.length);
const lead = rows.find((row) => row.textContent?.includes("Should we store OTEL agent spans"));
expect(lead).toBeDefined();
const failed = rows.find((row) => row.textContent?.includes("acme-404")) as HTMLElement;
expect(within(failed).getByLabelText("2 errors")).toBeInTheDocument();
expect(screen.getByText(`${runs.length} runs`)).toBeInTheDocument();
expect(screen.getByRole("columnheader", { name: "Cost" })).toBeInTheDocument();
expect(within(failed).getByText("—")).toBeInTheDocument();
});
it("shows the spend returned for a run", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({
...(traceList as TracePage),
data: [{ ...runs[0], spend: 0.025 }],
});
renderSection();
const row = await screen.findByTestId("agent-trace-row");
expect(within(row).getByText("$0.03")).toBeInTheDocument();
});
it("filters by input text and by trace id", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
await screen.findAllByTestId("agent-trace-row");
const search = screen.getByLabelText("Search runs");
fireEvent.change(search, { target: { value: "acme-404" } });
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(1);
const lead = runs.find((r) => r.name === "research_lead") as TraceSummary;
fireEvent.change(search, { target: { value: lead.trace_id.slice(0, 10) } });
const rows = screen.getAllByTestId("agent-trace-row");
expect(rows).toHaveLength(1);
expect(rows[0]).toHaveTextContent("Should we store OTEL agent spans");
});
it("uses recorded agent names for the column and filter even when services are shared", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({
...(traceList as TracePage),
data: [
...runs.slice(1).map((run) => ({ ...run, service: "shared-app", agent_names: ["research-agent"] })),
{ ...runs[0], service: "shared-app", agent_names: ["billing-agent", "review-agent"] },
],
});
const user = userEvent.setup();
renderSection();
await screen.findAllByTestId("agent-trace-row");
expect(screen.getByRole("columnheader", { name: "Agent" })).toBeInTheDocument();
const agentFilter = screen.getByRole("combobox", { name: "Filter by agent" });
expect(agentFilter).toHaveTextContent("All agents");
await chooseSelectOption(user, agentFilter, "billing-agent");
const rows = screen.getAllByTestId("agent-trace-row");
expect(rows).toHaveLength(1);
expect(rows[0]).toHaveTextContent("billing-agent");
expect(rows[0]).not.toHaveTextContent("shared-app");
await chooseSelectOption(user, agentFilter, "review-agent");
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(1);
await chooseSelectOption(user, agentFilter, "All agents");
expect(screen.getAllByTestId("agent-trace-row")).toHaveLength(runs.length);
});
it("shows each run's agent name with the logo of the SDK that produced it", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue({
...(traceList as TracePage),
data: [
{ ...runs[0], agent_names: ["research-bot"], frameworks: ["claude-agent-sdk", "claude-code"] },
{ ...runs[1], agent_names: [], frameworks: ["claude-code"] },
{ ...runs[2], frameworks: [] },
],
});
renderSection();
const [sdkRun, cliRun, plainRun] = await screen.findAllByTestId("agent-trace-row");
const agentCell = (row: HTMLElement) => within(row).getAllByRole("cell")[1];
expect(agentCell(sdkRun)).toHaveTextContent(/^research-bot$/);
expect(agentCell(sdkRun)).toHaveAttribute("title", "research-bot · Claude Agent SDK");
expect(within(sdkRun).getByRole("img", { name: "Claude Agent SDK logo", hidden: true })).toHaveAttribute(
"src",
expect.stringContaining("anthropic.svg"),
);
expect(agentCell(cliRun)).toHaveTextContent(/^Claude Code$/);
expect(within(plainRun).queryByRole("img", { hidden: true })).not.toBeInTheDocument();
expect(agentCell(plainRun)).toHaveTextContent((runs[2].agent_names ?? [runs[2].service]).join(", "));
});
describe("filterRuns", () => {
it("status filter 'Failed' keeps only runs with errors", () => {
const failed = filterRuns(runs, "", "all", "error");
expect(failed.length).toBeGreaterThan(0);
@ -332,151 +14,4 @@ describe("AgentTracesSection", () => {
expect(ok.every((r) => r.error_count === 0)).toBe(true);
expect(failed.length + ok.length).toBe(runs.length);
});
it("opens a run in a side drawer over the list and swaps runs without closing it", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
const drawer = screen.getByRole("complementary", { name: "Trace details" });
expect(within(drawer).getByTestId("run-view")).toHaveTextContent(`run ${runs[0].trace_id}`);
expect(screen.getByTestId("runs-table")).toBeInTheDocument();
expect(rows[0]).toHaveAttribute("aria-selected", "true");
fireEvent.click(rows[1]);
expect(screen.getByRole("complementary", { name: "Trace details" })).toBe(drawer);
expect(within(drawer).getByTestId("run-view")).toHaveTextContent(`run ${runs[1].trace_id}`);
expect(rows[1]).toHaveAttribute("aria-selected", "true");
expect(rows[0]).toHaveAttribute("aria-selected", "false");
});
it("closes the drawer when the open row is clicked again or Escape is pressed", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
fireEvent.click(rows[0]);
expect(rows[0]).toHaveAttribute("aria-selected", "false");
fireEvent.click(rows[1]);
fireEvent.keyDown(window, { key: "Escape" });
expect(rows[1]).toHaveAttribute("aria-selected", "false");
});
it("moves to the next and previous run with j / k and the header arrows", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderSection();
const rows = await screen.findAllByTestId("agent-trace-row");
fireEvent.click(rows[0]);
fireEvent.keyDown(window, { key: "j" });
expect(screen.getByTestId("run-view")).toHaveTextContent(`run ${runs[1].trace_id}`);
fireEvent.keyDown(window, { key: "k" });
expect(screen.getByTestId("run-view")).toHaveTextContent(`run ${runs[0].trace_id}`);
expect(screen.getByRole("button", { name: "Previous trace (K)" })).toBeDisabled();
fireEvent.click(screen.getByRole("button", { name: "Next trace (J)" }));
expect(screen.getByTestId("run-view")).toHaveTextContent(`run ${runs[1].trace_id}`);
});
it("plots every loaded run on the timeline", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWindowed();
await screen.findAllByTestId("agent-trace-row");
expect(screen.getByTestId("traces-timeline")).toBeInTheDocument();
const counts = bucketRunCounts();
expect(counts).toHaveLength(60);
expect(counts.reduce((a, b) => a + b, 0)).toBe(runs.length);
});
it("zooms by dragging, resizes and pans the bracket, and clears with Esc", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWindowed();
await screen.findAllByTestId("agent-trace-row");
const area = screen.getByTestId("timeline-area");
const x = (bucket: number) => bucket * 10 + 5;
const drag = (target: HTMLElement, from: number, to: number) => {
fireEvent.pointerDown(target, { clientX: x(from), pointerId: 1 });
fireEvent.pointerMove(area, { clientX: x(to), pointerId: 1 });
fireEvent.pointerUp(area, { clientX: x(to), pointerId: 1 });
};
const rowCount = () => screen.queryAllByTestId("agent-trace-row").length;
const withRuns = bucketRunCounts().flatMap((count, i) => (count > 0 ? [i] : []));
const first = withRuns[0];
// The pan below moves a [0, first] bracket to the far right; it must end up clear of every run.
expect(first).toBeGreaterThan(1);
expect(first).toBeLessThan(30);
drag(area, 0, 1);
expect(screen.getByTestId("timeline-selection")).toBeInTheDocument();
expect(rowCount()).toBe(0);
drag(screen.getByTestId("timeline-handle-hi"), 1, first);
expect(rowCount()).toBeGreaterThan(0);
drag(screen.getByTestId("timeline-selection"), 1, 1 - first);
expect(rowCount()).toBeGreaterThan(0);
drag(screen.getByTestId("timeline-selection"), 0, 59);
expect(rowCount()).toBe(0);
fireEvent.keyDown(screen.getByTestId("traces-timeline"), { key: "Escape" });
expect(screen.queryByTestId("timeline-selection")).not.toBeInTheDocument();
expect(rowCount()).toBe(runs.length);
});
});
describe("AgentTracesPage", () => {
beforeEach(() => {
testQueryClient.clear();
vi.mocked(agentTraceListCall).mockReset();
vi.mocked(apiClient.get).mockResolvedValue({ data: [] });
});
it("shows the actual range, switches presets from the popover, and toggles Live", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWithProviders(<AgentTracesPage accessToken="sk-test" />);
await screen.findByTestId("runs-table");
const trigger = screen.getByRole("button", { name: "Time range" });
expect(trigger).toHaveTextContent(/ to /);
expect(screen.getByTestId("traces-timeline")).toHaveTextContent("Total 1d");
fireEvent.click(trigger);
fireEvent.click(await screen.findByRole("menuitemradio", { name: "Last 7 days" }));
expect(await screen.findByText("Total 7d")).toBeInTheDocument();
const last = vi.mocked(agentTraceListCall).mock.calls.at(-1)?.[0];
expect((last?.endMs ?? 0) - (last?.startMs ?? 0)).toBeGreaterThanOrEqual(7 * 24 * 3600 * 1000 - 60_000);
const live = screen.getByRole("button", { name: "Live" });
expect(live).toHaveAttribute("aria-pressed", "true");
fireEvent.click(live);
expect(live).toHaveAttribute("aria-pressed", "false");
expect(screen.getByRole("button", { name: "Reset zoom" })).toBeDisabled();
});
it("keeps the time controls on an empty range the user picked, instead of showing onboarding", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWithProviders(<AgentTracesPage accessToken="sk-test" />);
await screen.findByTestId("runs-table");
vi.mocked(agentTraceListCall).mockResolvedValue({ ...(traceList as TracePage), data: [] });
fireEvent.click(screen.getByRole("button", { name: "Time range" }));
fireEvent.click(await screen.findByRole("menuitemradio", { name: "Last hour" }));
expect(await screen.findByText("No runs match these filters.")).toBeInTheDocument();
expect(screen.getByRole("button", { name: "Time range" })).toBeInTheDocument();
expect(screen.queryByTestId("tracing-setup-card")).not.toBeInTheDocument();
});
it("asks the proxy for the last 24 hours by default", async () => {
vi.mocked(agentTraceListCall).mockResolvedValue(traceList as TracePage);
renderWithProviders(<AgentTracesPage accessToken="sk-test" />);
await screen.findByTestId("runs-table");
const { startMs, endMs } = vi.mocked(agentTraceListCall).mock.calls[0][0];
expect(endMs - startMs).toBeGreaterThanOrEqual(24 * 3600 * 1000 - 60_000);
expect(endMs - startMs).toBeLessThan(24 * 3600 * 1000 + 120_000);
});
});

43
ui/litellm-dashboard/src/components/view_logs/TraceView/AgentTracesSection.tsx
View file

@ -1,4 +1,5 @@
"use client";
import { useLensDemo } from "@/components/lens/LensDemoContext";
import moment from "moment";
import { useMemo, useState } from "react";
@ -61,6 +62,7 @@ interface AgentTracesSectionProps {
onRunOpenChange?: (open: boolean) => void;
readOnly?: boolean;
canMintTracingKey?: boolean;
onDemo?: () => void;
}
function useTracingSetup(traces: AgentTracesResult, isActive: boolean, rangeChanged: boolean) {
@ -102,7 +104,9 @@ export function AgentTracesSection({
onRunOpenChange,
readOnly = false,
canMintTracingKey = false,
onDemo,
}: AgentTracesSectionProps) {
const demo = useLensDemo();
const [openTrace, setOpenTrace] = useState<TraceSummary | null>(null);
const [query, setQuery] = useState("");
const [agent, setAgent] = useState(ALL_AGENTS);
@ -160,10 +164,11 @@ export function AgentTracesSection({
checking: traces.isFetching,
};
if (setup.disabledDetail != null) return <TracingSetupCard detail={setup.disabledDetail} {...setupProps} />;
if (setup.disabledDetail != null)
return <TracingSetupCard detail={setup.disabledDetail} onDemo={onDemo} {...setupProps} />;
// Onboarding only on the first, default view; an empty range the user picked keeps its controls.
if (checkHistory && !history.error && history.data === false)
return <TracingSetupCard detail={null} {...setupProps} />;
return <TracingSetupCard detail={null} onDemo={onDemo} {...setupProps} />;
if (showSetup) {
return (
<div>
@ -185,11 +190,7 @@ export function AgentTracesSection({
return (
<div className="flex min-h-[560px] flex-1 flex-col overflow-hidden border-y border-border bg-card">
{checkHistory && <TraceHistoryError history={history} />}
{setup.received && (
<p role="status" className="border-b px-3 py-3 text-sm text-emerald-700 dark:text-emerald-400">
Traces received. Select a run to inspect it.
</p>
)}
<TracesReceived received={setup.received} />
<RunDrawer trace={openTrace} runs={runs} accessToken={accessToken} onSelect={openRun} />
<RunsToolbar
query={query}
@ -200,19 +201,25 @@ export function AgentTracesSection({
onAgentChange={setAgent}
onStatusChange={setStatus}
>
<Button variant="outline" size="sm" onClick={() => setShowSetup(true)} className="shrink-0 gap-1.5">
<ActiveDot />
Set up tracing
</Button>
{!demo && (
<Button variant="outline" size="sm" onClick={() => setShowSetup(true)} className="shrink-0 gap-1.5">
<ActiveDot />
Set up tracing
</Button>
)}
{timeControls && (
<TimeRangeControls
range={zoom ?? range}
rangeHours={timeControls.rangeHours}
onRangeHoursChange={(hours) => changeRange(hours, timeControls.onRangeHoursChange)}
live={isLiveTail}
showLive={!demo}
onLiveChange={timeControls.onLiveChange}
zoomed={zoom !== null}
onResetZoom={() => setZoom(null)}
onRefresh={() => {
setZoom(null);
checkTraces();
}}
refreshing={traces.isFetching}
/>
)}
</RunsToolbar>
@ -237,6 +244,16 @@ export function AgentTracesSection({
);
}
function TracesReceived({ received }: { received: boolean }) {
const demo = useLensDemo();
if (!received || demo) return null;
return (
<p role="status" className="border-b px-3 py-3 text-sm text-emerald-700 dark:text-emerald-400">
Traces received. Select a run to inspect it.
</p>
);
}
function RunsFooter({
count,
zoom,

17
ui/litellm-dashboard/src/components/view_logs/TraceView/DetailContent.tsx
View file

@ -1,4 +1,5 @@
"use client";
import { useLensDemo } from "@/components/lens/LensDemoContext";
import { useQuery, type UseQueryOptions } from "@tanstack/react-query";
import { useState } from "react";
@ -29,9 +30,15 @@ const errorReason = (headline: string): string => /^([A-Za-z_][\w.]*)\(/.exec(he
/** Shared lazy fetch of one span's full input / output / attributes. */
export function useSpanDetail(accessToken: string, traceId: string, spanId: string | null, traceRef?: string) {
const demo = useLensDemo();
const queryOptions: UseQueryOptions<SpanDetail, Error> = {
queryKey: ["agentTraceSpan", traceId, traceRef, spanId, accessToken],
queryFn: () => agentTraceSpanCall(accessToken, traceId, spanId as string, traceRef),
queryFn: () =>
demo
? demo.client.get<SpanDetail>(
`/v1/traces/${encodeURIComponent(traceId)}/spans/${encodeURIComponent(spanId as string)}`,
)
: agentTraceSpanCall(accessToken, traceId, spanId as string, traceRef),
enabled: spanId !== null,
staleTime: Infinity,
};
@ -146,11 +153,17 @@ interface DetailContentProps {
}
function DiagnosticContent({ accessToken, traceId, traceRef, span }: DetailContentProps) {
const demo = useLensDemo();
const [opened, setOpened] = useState(false);
const [cursor, setCursor] = useState<string | null>(null);
const queryOptions: UseQueryOptions<SpanErrorPage, Error> = {
queryKey: ["agentTraceSpanError", traceId, traceRef, span.span_id, accessToken, cursor],
queryFn: () => agentTraceSpanErrorCall(accessToken, traceId, span.span_id, { traceRef, cursor }),
queryFn: () =>
demo
? demo.client.get<SpanErrorPage>(
`/v1/traces/${encodeURIComponent(traceId)}/spans/${encodeURIComponent(span.span_id)}/error`,
)
: agentTraceSpanErrorCall(accessToken, traceId, span.span_id, { traceRef, cursor }),
enabled: opened,
staleTime: Infinity,
gcTime: 0,

17
ui/litellm-dashboard/src/components/view_logs/TraceView/DetailPane.tsx
View file

@ -1,4 +1,5 @@
"use client";
import { useLensDemo } from "@/components/lens/LensDemoContext";
import { PanelRightClose } from "lucide-react";
import { useState } from "react";
@ -104,6 +105,7 @@ function SpanPane({
onClose: () => void;
}) {
const [tab, setTab] = useState<Tab>("content");
const demo = useLensDemo();
const traceId = trace.summary.trace_id;
const detailQuery = useSpanDetail(
accessToken,
@ -162,9 +164,13 @@ function SpanPane({
</div>
<PaneFooter>
<CopyButton
value={agentHandoffText(traceId, span.span_id, trace.summary.trace_ref)}
value={
demo
? demo.copyTrace(traceId, span.span_id)
: agentHandoffText(traceId, span.span_id, trace.summary.trace_ref)
}
label="Copy step"
copiedLabel="Command copied"
copiedLabel={demo ? "Step copied" : "Command copied"}
/>
<div className="ml-auto flex items-center gap-3 text-[13px] text-trace-duration tabular-nums">
<Meta label="time" value={fmtMs(span.duration_ms)} />
@ -186,6 +192,7 @@ function GroupMetric({ label, value }: { label: string; value: string }) {
/** ×N group: rollup of every invocation plus the first failure's message. */
function GroupPane({ trace, row, onClose }: { trace: Trace; row: GroupRowData; onClose: () => void }) {
const demo = useLensDemo();
const tokens = row.members.reduce((sum, m) => sum + m.input_tokens + m.output_tokens, 0);
const firstFailure = row.members.find((m) => m.status === "error" && m.error);
return (
@ -222,7 +229,11 @@ function GroupPane({ trace, row, onClose }: { trace: Trace; row: GroupRowData; o
</div>
<PaneFooter>
<CopyButton
value={agentHandoffText(trace.summary.trace_id, (firstFailure ?? row.members[0]).span_id)}
value={
demo
? demo.copyTrace(trace.summary.trace_id, (firstFailure ?? row.members[0]).span_id)
: agentHandoffText(trace.summary.trace_id, (firstFailure ?? row.members[0]).span_id)
}
label="Copy group sample"
/>
</PaneFooter>

49
ui/litellm-dashboard/src/components/view_logs/TraceView/TimeRangeControls.tsx
View file

@ -34,31 +34,34 @@ interface TimeRangeControlsProps {
rangeHours: number;
onRangeHoursChange: (hours: number) => void;
live: boolean;
showLive?: boolean;
onLiveChange: (live: boolean) => void;
zoomed: boolean;
onResetZoom: () => void;
onRefresh: () => void;
refreshing: boolean;
}
/** Joined control group: reset zoom, the actual time range (opens presets), and Live. */
/** Joined control group: refresh, the actual time range (opens presets), and Live. */
export function TimeRangeControls({
range,
rangeHours,
onRangeHoursChange,
live,
showLive = true,
onLiveChange,
zoomed,
onResetZoom,
onRefresh,
refreshing,
}: TimeRangeControlsProps) {
return (
<div className="flex items-center gap-1.5">
<button
type="button"
onClick={onResetZoom}
disabled={!zoomed}
aria-label="Reset zoom"
className="inline-flex size-7 items-center justify-center rounded-md border border-border text-info hover:bg-info/10 disabled:text-muted-foreground disabled:opacity-40 disabled:hover:bg-transparent"
onClick={onRefresh}
aria-label="Refresh"
title="Refresh"
aria-busy={refreshing}
className="inline-flex size-7 items-center justify-center rounded-md border border-border text-muted-foreground hover:text-foreground hover:bg-muted/60"
>
<RotateCcw className="size-3.5" />
<RotateCcw className={cn("size-3.5", refreshing && "animate-spin")} />
</button>
<div className="flex items-center divide-x divide-border overflow-hidden rounded-md border border-border bg-card">
<DropdownMenu>
@ -83,18 +86,20 @@ export function TimeRangeControls({
</DropdownMenuRadioGroup>
</DropdownMenuContent>
</DropdownMenu>
<button
type="button"
aria-pressed={live}
onClick={() => onLiveChange(!live)}
className={cn(
SEGMENT,
live ? "bg-info/10 text-info hover:bg-info/15" : "text-muted-foreground hover:text-foreground",
)}
>
{live ? <Pause className="size-3.5" /> : <Play className="size-3.5" />}
Live
</button>
{showLive && (
<button
type="button"
aria-pressed={live}
onClick={() => onLiveChange(!live)}
className={cn(
SEGMENT,
live ? "bg-info/10 text-info hover:bg-info/15" : "text-muted-foreground hover:text-foreground",
)}
>
{live ? <Pause className="size-3.5" /> : <Play className="size-3.5" />}
Live
</button>
)}
</div>
</div>
);

17
ui/litellm-dashboard/src/components/view_logs/TraceView/TraceDrawer.tsx
View file

@ -1,4 +1,5 @@
"use client";
import { useLensDemo } from "@/components/lens/LensDemoContext";
import { useQuery } from "@tanstack/react-query";
import { ArrowLeft, Check, Copy } from "lucide-react";
@ -74,6 +75,7 @@ const toggle = (set: ReadonlySet<string>, id: string): Set<string> => {
};
function CopyForAgent({ traceId, traceRef }: { traceId: string; traceRef?: string }) {
const demo = useLensDemo();
const [copied, setCopied] = useState(false);
useEffect(() => {
if (!copied) return;
@ -86,11 +88,16 @@ function CopyForAgent({ traceId, traceRef }: { traceId: string; traceRef?: strin
size="xs"
className="h-7 shrink-0 gap-1.5 rounded-md text-[12px] shadow-none"
onClick={async () =>
setCopied(await copyToClipboard(agentHandoffText(traceId, null, traceRef), "Command copied"))
setCopied(
await copyToClipboard(
demo ? demo.copyTrace(traceId) : agentHandoffText(traceId, null, traceRef),
demo ? "Trace copied" : "Command copied",
),
)
}
>
{copied ? <Check className="size-3" /> : <Copy className="size-3" />}
{copied ? "Command copied" : "Copy for agent"}
{copied ? "Copied" : "Copy for agent"}
</Button>
);
}
@ -277,9 +284,13 @@ interface RunViewProps {
/** One agent run: header with totals and "Copy for agent", span tree on the left, span details on the right. */
export function RunView({ traceId, traceRef, initialSpanId, accessToken, onBack, embedded = false }: RunViewProps) {
const demo = useLensDemo();
const traceQuery = useQuery({
queryKey: ["agentTrace", traceId, traceRef, accessToken],
queryFn: () => agentTraceCall(accessToken, traceId, traceRef),
queryFn: () =>
demo
? demo.client.get<Trace>(`/v1/traces/${encodeURIComponent(traceId)}`)
: agentTraceCall(accessToken, traceId, traceRef),
staleTime: 30_000,
});
const trace = traceQuery.data;

70
ui/litellm-dashboard/src/components/view_logs/TraceView/TracePreview.tsx
View file

@ -1,70 +0,0 @@
"use client";
import { cn } from "@/lib/cva.config";
import previewTrace from "./previewTrace.json";
import { SpanIcon } from "./SpanIcon";
import type { SpanType } from "./traceTypes";
import { fmtMs } from "./traceUtils";
interface PreviewRow {
id: string;
name: string;
type: SpanType;
model: string | null;
depth: number;
start_offset_ms: number;
duration_ms: number;
error: boolean;
}
const preview = previewTrace as {
name: string;
input_preview: string;
span_count: number;
duration_ms: number;
rows: PreviewRow[];
};
export function TracePreview() {
const total = preview.duration_ms;
return (
<div
aria-hidden
data-testid="trace-preview"
className="pointer-events-none overflow-hidden rounded-lg border border-border bg-background shadow-sm select-none"
>
<div className="flex items-center gap-3 border-b border-border px-4 py-2.5">
<SpanIcon type="agent" size="md" />
<span className="text-[13px] font-medium text-foreground">{preview.name}</span>
<span className="truncate text-[12px] text-muted-foreground">{preview.input_preview}</span>
<span className="ml-auto shrink-0 font-mono text-[11px] text-muted-foreground">
{preview.span_count} steps · {fmtMs(total)}
</span>
</div>
<div className="relative">
{preview.rows.map((row) => (
<div key={row.id} className="flex h-8 items-center gap-2 border-b border-border/60 px-4 last:border-b-0">
<div className="flex min-w-0 flex-1 items-center gap-2" style={{ paddingLeft: row.depth * 16 }}>
<SpanIcon type={row.type} model={row.model} error={row.error} size="sm" />
<span className="truncate text-[12.5px] text-foreground">{row.name}</span>
</div>
<div className="relative h-1.5 w-[40%] shrink-0 rounded-full bg-muted">
<div
className={cn("absolute h-full rounded-full", row.error ? "bg-destructive" : "bg-trace-brand/70")}
style={{
left: `${(row.start_offset_ms / total) * 100}%`,
width: `${Math.max((row.duration_ms / total) * 100, 0.6)}%`,
}}
/>
</div>
<span className="w-14 shrink-0 text-right font-mono text-[11px] text-muted-foreground">
{fmtMs(row.duration_ms)}
</span>
</div>
))}
<div className="pointer-events-none absolute inset-x-0 bottom-0 h-16 bg-gradient-to-t from-background to-transparent" />
</div>
</div>
);
}

111
ui/litellm-dashboard/src/components/view_logs/TraceView/TracingSetupCard.test.tsx → ui/litellm-dashboard/src/components/view_logs/TraceView/TracingSetupCard.integration.test.tsx
View file

@ -12,6 +12,7 @@ import {
tracingEnvSnippet,
TracingSetupCard,
} from "./TracingSetupCard";
import { FRAMEWORKS } from "./tracingSetupGuides";
import type { Trace } from "./traceTypes";
vi.mock("../../networking", () => ({
@ -31,6 +32,7 @@ const renderCard = (
onCheck?: () => void;
readOnly?: boolean;
canMintTracingKey?: boolean;
onDemo?: () => void;
} = {},
) => {
const onOpenTrace = vi.fn();
@ -38,6 +40,7 @@ const renderCard = (
<TracingSetupCard
detail={props.detail ?? null}
connected={props.connected}
onDemo={props.onDemo}
onCheck={props.onCheck}
readOnly={props.readOnly}
canMintTracingKey={props.canMintTracingKey ?? true}
@ -51,69 +54,98 @@ const renderCard = (
beforeEach(() => vi.clearAllMocks());
describe("TracingSetupCard", () => {
it("shows agent connection guidance and labels the example run as sample data", async () => {
it("offers the interactive demo while waiting for the first trace", async () => {
const user = userEvent.setup();
const { card } = renderCard();
const onDemo = vi.fn();
const { card } = renderCard({ onDemo });
expect(screen.getByRole("heading", { name: "Connect your agent" })).toBeVisible();
expect(screen.getByText("Tracing enabled")).toBeVisible();
expect(screen.getByText("Waiting for your first trace")).toBeVisible();
expect(screen.getByTestId("trace-preview")).toBeVisible();
expect(card).toHaveTextContent("sample data, not your runs");
await user.click(screen.getByRole("button", { name: "Preview sample" }));
expect(onDemo).toHaveBeenCalledOnce();
expect(sendOtlpTraceCall).not.toHaveBeenCalled();
expect(card).not.toHaveTextContent("store: clickhouse");
await user.click(screen.getByText("Set up manually"));
expect(screen.getByText(/^export OTEL_EXPORTER_OTLP_ENDPOINT=http:\/\/proxy.test/)).toBeVisible();
expect(screen.getByText(/^export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=/)).toBeVisible();
expect(card).not.toHaveTextContent(/langsmith/i);
});
it("lists the OTEL endpoints before the framework picker, each copyable", async () => {
it("keeps connection details visible and copies the full trace endpoint", async () => {
const user = userEvent.setup();
const { card } = renderCard();
const text = card.textContent ?? "";
expect(text.indexOf("OpenTelemetry (OTEL) endpoints")).toBeLessThan(text.indexOf("Your agent framework"));
renderCard();
expect(screen.getByRole("combobox", { name: "Your agent framework" })).toBeVisible();
expect(screen.getByRole("button", { name: "Copy http://proxy.test/v1/traces" })).toBeVisible();
await user.click(screen.getByRole("button", { name: "Copy http://proxy.test/v1/traces" }));
expect(copyToClipboard).toHaveBeenLastCalledWith("http://proxy.test/v1/traces");
});
it("hides the sample preview once traces are arriving", () => {
it("shows connection guidance for another agent without a demo", () => {
renderCard({ connected: true });
expect(screen.getByRole("heading", { name: "Connect another agent" })).toBeVisible();
expect(screen.queryByTestId("trace-preview")).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Preview sample" })).not.toBeInTheDocument();
});
it("builds the coding agent command for the selected framework and keeps both manual installers", async () => {
const user = userEvent.setup();
renderCard();
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Your agent framework" }), "CrewAI");
const prompt = codingAgentPrompt("http://proxy.test", {
label: "CrewAI",
packages: "crewai openinference-instrumentation-crewai",
});
const commandText = () => screen.getByText(/^claude |^codex /).textContent;
expect(commandText()).toBe(codingAgentCommand("Claude Code", prompt));
const prompt = codingAgentPrompt(
"http://proxy.test",
FRAMEWORKS.find((guide) => guide.id === "crewai")!,
"openai/gpt-6-sol",
);
expect(screen.getByText(/^claude /)).not.toBeVisible();
await user.click(screen.getByRole("button", { name: "Copy setup command" }));
expect(copyToClipboard).toHaveBeenLastCalledWith(codingAgentCommand("Claude Code", prompt));
await user.click(screen.getByRole("tab", { name: "Codex" }));
expect(commandText()).toBe(codingAgentCommand("Codex", prompt));
await user.click(screen.getByRole("button", { name: "Copy setup command" }));
expect(copyToClipboard).toHaveBeenLastCalledWith(codingAgentCommand("Codex", prompt));
await user.click(screen.getByText("View command"));
expect(screen.getByText(/^codex /)).toBeVisible();
expect(screen.getByText(/^codex /)).toHaveTextContent(codingAgentCommand("Codex", prompt), {
normalizeWhitespace: false,
});
await user.click(screen.getByText("Set up manually"));
expect(screen.getByText(/pip install -U opentelemetry-distro/)).toHaveTextContent(
expect(screen.getByText(/^pip install opentelemetry-distro/)).toHaveTextContent(
"crewai openinference-instrumentation-crewai",
);
await user.click(screen.getByRole("tab", { name: "uv" }));
expect(screen.getByText(/uv add opentelemetry-distro/)).toHaveTextContent(
expect(screen.getByText(/^uv add opentelemetry-distro/)).toHaveTextContent(
"crewai openinference-instrumentation-crewai",
);
});
it("uses npm and a CommonJS-safe TypeScript entrypoint for the Vercel AI SDK", async () => {
it("uses the selected framework's tracing and agent name without asking for a model", async () => {
const user = userEvent.setup();
const { card } = renderCard();
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Your agent framework" }), "Vercel AI SDK");
await user.click(screen.getByText("Set up manually"));
expect(card).toHaveTextContent("npm install ai @ai-sdk/openai-compatible @vercel/otel");
expect(card).toHaveTextContent("my_agent.ts");
expect(card).not.toHaveTextContent("opentelemetry-instrument python");
const quickstart = screen.getByText(/registerOTel\(\{/).textContent ?? "";
expect(quickstart).toContain("async function main()");
expect(quickstart.split("\n").filter((line) => /^(const|let) .*= await /.test(line))).toEqual([]);
expect(screen.queryByRole("combobox", { name: "Model" })).not.toBeInTheDocument();
expect(card).toHaveTextContent("npm install ai @ai-sdk/otel");
expect(card).toHaveTextContent('const AGENT_NAME = "research_agent"');
expect(card).toHaveTextContent("functionId: AGENT_NAME");
expect(card).toHaveTextContent("Use a model configured on this proxy.");
expect(screen.getByText(/^import \{ createOpenAICompatible/)).toHaveTextContent(
'const model = litellm("openai/gpt-6-sol")',
);
expect(card).toHaveTextContent('baseURL: "http://proxy.test/v1"');
});
it("keeps plugin model settings and uses a generated tracing key only for tracing", async () => {
const user = userEvent.setup();
vi.mocked(apiClient.post).mockResolvedValue({ key: SECRET });
const { card } = renderCard();
await chooseSelectOption(user, screen.getByRole("combobox", { name: "Your agent framework" }), "Hermes");
await user.click(screen.getByText("Set up manually"));
expect(screen.queryByRole("combobox", { name: "Model" })).not.toBeInTheDocument();
expect(card).toHaveTextContent("Keep your existing model settings");
await user.click(screen.getByRole("button", { name: "Generate tracing key" }));
await screen.findByText("Your tracing key");
expect(card).toHaveTextContent("gen_ai.agent.name: research_agent");
expect(card).toHaveTextContent("endpoint: http://proxy.test/v1/traces");
expect(card).toHaveTextContent('Authorization: "Bearer ${LITELLM_TRACING_KEY}"');
expect(card).not.toHaveTextContent(SECRET);
});
it("hides the actions a read-only viewer cannot perform", () => {
@ -121,7 +153,7 @@ describe("TracingSetupCard", () => {
expect(screen.queryByRole("button", { name: "Send a test trace" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Generate tracing key" })).not.toBeInTheDocument();
expect(card).toHaveTextContent("ask a proxy admin for one");
expect(card).toHaveTextContent("OpenTelemetry (OTEL) endpoints");
expect(card).toHaveTextContent("Connection details");
});
it("offers a scoped tracing key only to callers allowed to set key routes", () => {
@ -135,7 +167,7 @@ describe("TracingSetupCard", () => {
const user = userEvent.setup();
vi.mocked(apiClient.post).mockResolvedValue({ key: SECRET });
const { card } = renderCard();
await user.click(screen.getByText("Set up manually"));
await user.click(screen.getByRole("button", { name: "Generate tracing key" }));
expect(await screen.findByText("Your tracing key")).toBeVisible();
@ -193,24 +225,23 @@ describe("TracingSetupCard", () => {
});
describe("setup snippets", () => {
it("uses OTLP HTTP with the proxy base URL and reads the key from the environment", () => {
it("uses the instance trace endpoint and keeps tracing and inference keys separate", () => {
const env = tracingEnvSnippet("http://proxy.test");
expect(env).toContain("OTEL_EXPORTER_OTLP_ENDPOINT=http://proxy.test\n");
expect(env).toContain("OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf");
expect(env).not.toContain("/v1/traces");
expect(env).toContain('OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://proxy.test/v1/traces"');
expect(env).toContain('OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"');
expect(env).not.toContain("export LITELLM_API_KEY=");
expect(env).toContain("Bearer $LITELLM_API_KEY");
const withKey = tracingEnvSnippet("http://proxy.test", SECRET);
expect(withKey).toContain(`export LITELLM_TRACING_KEY=${SECRET}\n`);
expect(withKey).toContain('OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $LITELLM_TRACING_KEY"');
expect(withKey).not.toContain("LITELLM_API_KEY");
expect(withKey).toContain('OTEL_EXPORTER_OTLP_TRACES_HEADERS="Authorization=Bearer $LITELLM_TRACING_KEY"');
expect(withKey).not.toContain("export LITELLM_API_KEY=");
const prompt = codingAgentPrompt("http://proxy.test", { label: "LangChain", packages: "langchain" });
expect(prompt).toContain("base_url=http://proxy.test/v1");
expect(prompt).toContain("opentelemetry-distro opentelemetry-exporter-otlp-proto-http langchain");
expect(prompt).toContain("OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf");
const prompt = codingAgentPrompt("http://proxy.test", FRAMEWORKS[0], "openai/gpt-6-sol");
expect(prompt).toContain('OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://proxy.test/v1/traces"');
expect(prompt).toContain("Keep the existing model configuration");
expect(prompt).toContain('AGENT_NAME = "research_agent"');
expect(prompt).toContain("name=AGENT_NAME");
expect(prompt).toContain("Lens > Traces");
expect(prompt).not.toMatch(/langsmith/i);
});
it("builds a shell-safe command for each coding agent", () => {

498
ui/litellm-dashboard/src/components/view_logs/TraceView/TracingSetupCard.tsx
View file

@ -4,31 +4,23 @@ import { ArrowRight, ArrowUpRight, Check, Copy, KeyRound, Loader2, Send } from "
import { useState } from "react";
import { cn } from "@/lib/cva.config";
import { LensPreviewButton } from "@/components/lens/LensPreviewButton";
import { Button } from "@/components/ui/button";
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";
import { copyToClipboard } from "@/utils/dataUtils";
import anthropicLogo from "../../../../public/assets/logos/anthropic.svg";
import crewaiLogo from "../../../../public/assets/logos/crewai-color.svg";
import langchainLogo from "../../../../public/assets/logos/langchain.svg";
import langgraphLogo from "../../../../public/assets/logos/langgraph-color.svg";
import llamaindexLogo from "../../../../public/assets/logos/llamaindex-color.svg";
import openaiAgentsLogo from "../../../../public/assets/logos/openai-agents.svg";
import openaiLogo from "../../../../public/assets/logos/openai_small.svg";
import otelLogo from "../../../../public/assets/logos/opentelemetry.svg";
import pydanticAiLogo from "../../../../public/assets/logos/pydantic-ai-color.svg";
import vercelLogo from "../../../../public/assets/logos/vercel.svg";
import { agentTraceCall, apiClient, getProxyBaseUrl, sendOtlpTraceCall } from "../../networking";
import { ActiveDot } from "./ActiveDot";
import { sampleTraceExport } from "./sampleTrace";
import { TracePreview } from "./TracePreview";
import { FRAMEWORKS, frameworkSnippet, type FrameworkGuide } from "./tracingSetupGuides";
import type { TraceSummary } from "./traceTypes";
const COPIED_RESET_MS = 1500;
const DOCS_URL = "https://docs.litellm.ai/docs/proxy/lens";
const OTEL_BASE_PACKAGES = "opentelemetry-distro opentelemetry-exporter-otlp-proto-http";
const PY_RUN_SNIPPET = "opentelemetry-instrument python my_agent.py";
const TS_RUN_SNIPPET = "npx tsx my_agent.ts";
const EXAMPLE_MODEL = "openai/gpt-6-sol";
const SAMPLE_TRACE_POLL_MS = 1000;
export const TRACING_KEY_REQUEST = {
key_alias: "Agent tracing",
@ -41,177 +33,36 @@ type Installer = "pip" | "uv";
type CodingAgent = "Claude Code" | "Codex";
const PY_INSTALL: Record<Installer, (packages: string) => string> = {
pip: (packages) => `pip install -U ${packages}`,
pip: (packages) => `pip install ${packages}`,
uv: (packages) => `uv add ${packages}`,
};
interface FrameworkGuide {
id: string;
label: string;
logo: string;
packages: string;
quickstart: string;
typescript?: boolean;
}
const FRAMEWORKS: readonly FrameworkGuide[] = [
{
id: "deep-agents",
label: "Deep Agents",
logo: langgraphLogo.src,
packages: "deepagents langchain-openai openinference-instrumentation-langchain",
quickstart: `from deepagents import create_deep_agent
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="claude-sonnet-4-5", base_url="{PROXY}/v1", api_key=os.environ["LITELLM_API_KEY"])
agent = create_deep_agent(model=llm, tools=[], system_prompt="You are a careful researcher.")
agent.invoke({"messages": [{"role": "user", "content": "What is LiteLLM?"}]})`,
},
{
id: "vercel-ai-sdk",
label: "Vercel AI SDK",
logo: vercelLogo.src,
typescript: true,
packages: "ai @ai-sdk/openai-compatible @vercel/otel @opentelemetry/api",
quickstart: `import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { registerOTel } from "@vercel/otel";
import { generateText } from "ai";
registerOTel({ serviceName: process.env.OTEL_SERVICE_NAME ?? "my-agent" });
const litellm = createOpenAICompatible({ name: "litellm", baseURL: "{PROXY}/v1", apiKey: process.env.LITELLM_API_KEY });
async function main() {
const { text } = await generateText({
model: litellm("claude-sonnet-4-5"),
prompt: "What is LiteLLM?",
experimental_telemetry: { isEnabled: true, functionId: "my_agent" },
});
console.log(text);
}
main();`,
},
{
id: "langgraph",
label: "LangGraph",
logo: langgraphLogo.src,
packages: "langgraph langchain-openai openinference-instrumentation-langchain",
quickstart: `from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="claude-sonnet-4-5", base_url="{PROXY}/v1", api_key=os.environ["LITELLM_API_KEY"])
agent = create_agent(model=llm, tools=[], name="my_agent")
agent.invoke({"messages": [{"role": "user", "content": "What is LiteLLM?"}]})`,
},
{
id: "langchain",
label: "LangChain",
logo: langchainLogo.src,
packages: "langchain langchain-openai openinference-instrumentation-langchain",
quickstart: `from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="claude-sonnet-4-5", base_url="{PROXY}/v1", api_key=os.environ["LITELLM_API_KEY"])
llm.invoke("What is LiteLLM?")`,
},
{
id: "openai-agents",
label: "OpenAI Agents SDK",
logo: openaiAgentsLogo.src,
packages: "openai-agents openinference-instrumentation-openai-agents",
quickstart: `from agents import Agent, OpenAIChatCompletionsModel, Runner
from openai import AsyncOpenAI
client = AsyncOpenAI(base_url="{PROXY}/v1", api_key=os.environ["LITELLM_API_KEY"])
agent = Agent(name="my_agent", model=OpenAIChatCompletionsModel(model="claude-sonnet-4-5", openai_client=client))
print(Runner.run_sync(agent, "What is LiteLLM?").final_output)`,
},
{
id: "crewai",
label: "CrewAI",
logo: crewaiLogo.src,
packages: "crewai openinference-instrumentation-crewai",
quickstart: `from crewai import LLM, Agent, Crew, Task
llm = LLM(model="openai/claude-sonnet-4-5", base_url="{PROXY}/v1", api_key=os.environ["LITELLM_API_KEY"])
agent = Agent(role="Researcher", goal="Answer questions", backstory="", llm=llm)
task = Task(description="What is LiteLLM?", expected_output="A short answer", agent=agent)
Crew(agents=[agent], tasks=[task]).kickoff()`,
},
{
id: "pydantic-ai",
label: "Pydantic AI",
logo: pydanticAiLogo.src,
packages: "pydantic-ai openinference-instrumentation-pydantic-ai",
quickstart: `from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider
provider = OpenAIProvider(base_url="{PROXY}/v1", api_key=os.environ["LITELLM_API_KEY"])
agent = Agent(OpenAIModel("claude-sonnet-4-5", provider=provider), name="my_agent", instrument=True)
print(agent.run_sync("What is LiteLLM?").output)`,
},
{
id: "llamaindex",
label: "LlamaIndex",
logo: llamaindexLogo.src,
packages: "llama-index llama-index-llms-openai-like openinference-instrumentation-llama-index",
quickstart: `from llama_index.llms.openai_like import OpenAILike
llm = OpenAILike(model="claude-sonnet-4-5", api_base="{PROXY}/v1", api_key=os.environ["LITELLM_API_KEY"], is_chat_model=True)
print(llm.complete("What is LiteLLM?"))`,
},
{
id: "otel",
label: "OpenTelemetry",
logo: otelLogo.src,
packages: "",
quickstart: `# Any OTEL SDK works. Use the gen_ai.* semantic conventions:
# gen_ai.operation.name, gen_ai.agent.name, gen_ai.response.id, gen_ai.usage.*
from opentelemetry import trace
tracer = trace.get_tracer("my-agent")
attrs = {"gen_ai.operation.name": "invoke_agent", "gen_ai.agent.name": "my_agent"}
with tracer.start_as_current_span("my_agent", attributes=attrs):
...`,
},
];
const installPackages = (guide: Pick<FrameworkGuide, "packages" | "typescript">): string =>
guide.typescript ? guide.packages : [OTEL_BASE_PACKAGES, guide.packages].filter(Boolean).join(" ");
/** The endpoint is the proxy base URL: OTLP exporters append /v1/traces themselves. */
export const tracingEnvSnippet = (proxyUrl: string, tracingKey: string | null = null): string =>
[
...(tracingKey ? [`export LITELLM_TRACING_KEY=${tracingKey}`] : []),
`export OTEL_EXPORTER_OTLP_ENDPOINT=${proxyUrl}`,
"export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf",
`export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $${tracingKey ? "LITELLM_TRACING_KEY" : "LITELLM_API_KEY"}"`,
"export OTEL_SERVICE_NAME=my-agent",
`export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="${proxyUrl}/v1/traces"`,
`export OTEL_EXPORTER_OTLP_TRACES_HEADERS="Authorization=Bearer $${tracingKey ? "LITELLM_TRACING_KEY" : "LITELLM_API_KEY"}"`,
'export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"',
'export OTEL_METRICS_EXPORTER="none"',
'export OTEL_LOGS_EXPORTER="none"',
].join("\n");
export const codingAgentPrompt = (
proxyUrl: string,
guide: Pick<FrameworkGuide, "label" | "packages" | "typescript">,
): string =>
export const codingAgentPrompt = (proxyUrl: string, guide: FrameworkGuide, model: string): string =>
[
`Send this ${guide.label} project's OpenTelemetry traces to LiteLLM.`,
"",
`1. Add these dependencies: ${installPackages(guide)}`,
"2. Set these env vars wherever the project loads config (.env, settings, deployment manifests):",
` OTEL_EXPORTER_OTLP_ENDPOINT=${proxyUrl}`,
" OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf",
' OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $LITELLM_API_KEY"',
" OTEL_SERVICE_NAME=<a short name for this agent>",
guide.typescript
? "3. Call registerOTel() from @vercel/otel at startup and pass experimental_telemetry: { isEnabled: true } to every AI SDK call."
: "3. Start the app through OTEL auto-instrumentation: opentelemetry-instrument <existing start command>.",
`4. Point every LLM client at LiteLLM: base_url=${proxyUrl}/v1, api key from LITELLM_API_KEY.`,
"5. Give each agent and subagent a name so runs are easy to read.",
"6. Run the agent once and confirm the run shows up in the LiteLLM UI under Lens > Traces.",
"",
"Never hardcode the key. Read it from LITELLM_API_KEY.",
].join("\n");
"Keep the existing model configuration, authentication, and application behavior. Never hardcode a key; read it from LITELLM_API_KEY.",
"Set the trace destination wherever this project loads environment variables:",
tracingEnvSnippet(proxyUrl),
guide.install ?? `Install and enable the ${guide.plugin?.label}: ${guide.plugin?.url}`,
guide.plugin?.instruction ??
"Initialize OpenTelemetry before creating the agent. If the app already configures a tracer provider, keep it and point its exporter at the destination above instead.",
"Adapt this example to the existing application, replacing research_agent with the agent's name:",
frameworkSnippet(guide, proxyUrl, model),
guide.note ?? "",
"Run the agent once and confirm its named run appears in Lens > Traces.",
]
.filter(Boolean)
.join("\n\n");
const shellQuote = (value: string): string => `'${value.replaceAll("'", "'\\''")}'`;
@ -222,10 +73,9 @@ export const maskSecret = (secret: string): string =>
secret.length > 10 ? `${secret.slice(0, 5)}${"•".repeat(16)}${secret.slice(-4)}` : "•".repeat(secret.length);
export const otlpEndpoints = (proxyUrl: string): readonly (readonly [string, string, boolean])[] => [
["Traces endpoint (POST)", `${proxyUrl}/v1/traces`, true],
["OTEL_EXPORTER_OTLP_ENDPOINT", proxyUrl, true],
["Traces endpoint", `${proxyUrl}/v1/traces`, true],
["Auth header", "Authorization: Bearer <LiteLLM virtual key>", true],
["Protocol", "OTLP/HTTP, protobuf or JSON (gRPC not supported)", false],
["Protocol", "OTLP/HTTP (protobuf or JSON)", false],
];
export const PROXY_CONFIG_SNIPPET = [
@ -270,8 +120,8 @@ function CodeBlock({
</div>
<pre
className={cn(
"px-4 py-3 font-mono text-[12.5px] leading-6 text-foreground",
wrap ? "whitespace-pre-wrap" : "overflow-x-auto",
"m-0 px-4 py-3 font-mono text-xs leading-5 text-foreground",
wrap ? "whitespace-pre-wrap break-words" : "overflow-x-auto",
)}
>
<code>{display}</code>
@ -322,7 +172,7 @@ function LineTabs<T extends string>({
function Step({ title, children }: { title: React.ReactNode; children: React.ReactNode }) {
return (
<section className="mt-6">
<h3 className="mb-2 text-[13px] font-medium text-foreground">{title}</h3>
<h3 className="mb-3 text-sm font-medium text-foreground">{title}</h3>
{children}
</section>
);
@ -395,7 +245,7 @@ function SendTestTrace({
const busy = state.kind === "sending" || state.kind === "waiting";
return (
<div className="flex flex-wrap items-center gap-3">
<Button onClick={() => void send()} disabled={busy}>
<Button variant="outline" onClick={() => void send()} disabled={busy}>
{busy ? (
<Loader2 aria-hidden="true" className="size-4 animate-spin" />
) : (
@ -421,20 +271,18 @@ function TraceReceipt({
}) {
const missingAfterCheck = checked && !checking && !connected;
return (
<Step title="Run your agent once">
<p className="text-sm leading-6 text-muted-foreground">
Keep this page open. Your runs will appear here when LiteLLM receives them.
</p>
<div className="mt-4 flex flex-wrap items-center gap-3">
<section className="mt-6 border-t pt-6" aria-label="Trace connection status">
<div className="flex flex-wrap items-center justify-between gap-3">
<div>
<p role="status" className="text-sm font-medium">
{connected ? "Traces connected" : "Waiting for your first trace"}
</p>
<p className="mt-1 text-sm text-muted-foreground">Run your agent once. Its traces will appear here.</p>
</div>
<Button variant="outline" onClick={onCheck} disabled={checking}>
{checking && <Loader2 aria-hidden="true" className="size-4 animate-spin" />}
{checking ? "Checking…" : "Check for traces"}
</Button>
{!connected && (
<p role="status" className="text-sm text-muted-foreground">
Waiting for your first trace
</p>
)}
</div>
{missingAfterCheck && (
<p className="mt-3 text-sm leading-6 text-muted-foreground">
@ -442,7 +290,7 @@ function TraceReceipt({
logs for export errors.
</p>
)}
</Step>
</section>
);
}
@ -526,19 +374,19 @@ function EndpointValue({ value }: { value: string }) {
);
}
function Endpoints({ proxyUrl }: { proxyUrl: string }) {
function Endpoints({ proxyUrl, children }: { proxyUrl: string; children?: React.ReactNode }) {
return (
<section className="mt-6" aria-labelledby="otel-endpoints">
<h3 id="otel-endpoints" className="inline-flex items-center gap-2 text-sm font-medium">
<img src={otelLogo.src} alt="" className="size-5" />
OpenTelemetry (OTEL) endpoints
</h3>
<p className="mt-1 text-sm text-muted-foreground">
Point any OpenTelemetry exporter here. The framework guides below set these for you.
</p>
<dl className="mt-3 grid gap-x-8 gap-y-3 rounded-md border border-l-2 border-border border-l-trace-brand bg-muted/30 px-4 py-3 md:grid-cols-2">
<div className="flex flex-wrap items-center justify-between gap-3">
<h3 id="otel-endpoints" className="inline-flex items-center gap-2 text-sm font-medium">
<img src={otelLogo.src} alt="" className="size-5" />
Connection details
</h3>
{children}
</div>
<dl className="mt-3 grid gap-x-6 gap-y-3 rounded-md border bg-muted/30 p-4 md:grid-cols-2">
{otlpEndpoints(proxyUrl).map(([label, value, copyable]) => (
<div key={label} className="min-w-0">
<div key={label} className={cn("min-w-0", label === "Traces endpoint" && "md:col-span-2")}>
<dt className="text-[12px] text-muted-foreground">{label}</dt>
<dd className="mt-0.5">
{copyable ? <EndpointValue value={value} /> : <span className="text-[12.5px]">{value}</span>}
@ -574,7 +422,7 @@ interface ConnectAgentProps {
function EnableTracing({ checked, checking, onCheck }: { checked: boolean; checking: boolean; onCheck: () => void }) {
return (
<>
<Step title="Enable tracing on the proxy">
<Step title="Proxy configuration">
<p className="mb-3 text-sm leading-6 text-muted-foreground">
Set your ClickHouse URL, add this to config.yaml, then restart the proxy. Ask your proxy administrator if you
don’t manage this deployment.
@ -594,14 +442,68 @@ function EnableTracing({ checked, checking, onCheck }: { checked: boolean; check
Tracing is still unavailable. Check that the configuration was applied to this proxy and it has restarted.
</p>
)}
<Button className="mt-6" onClick={onCheck} disabled={checking}>
{checking && <Loader2 aria-hidden="true" className="size-4 animate-spin" />}
{checking ? "Checking…" : "Check setup"}
</Button>
<div className="mt-6 border-t pt-6">
<Button onClick={onCheck} disabled={checking}>
{checking && <Loader2 aria-hidden="true" className="size-4 animate-spin" />}
{checking ? "Checking…" : "Check setup"}
</Button>
</div>
</>
);
}
function CodingAgentSetup({ proxyUrl, guide, model }: { proxyUrl: string; guide: FrameworkGuide; model: string }) {
const [codingAgent, setCodingAgent] = useState<CodingAgent>("Claude Code");
const [copied, setCopied] = useState<string | null>(null);
const command = codingAgentCommand(codingAgent, codingAgentPrompt(proxyUrl, guide, model));
const copy = async () => {
if (await copyToClipboard(command)) {
setCopied(command);
window.setTimeout(() => setCopied(null), COPIED_RESET_MS);
}
};
return (
<section className="mt-6" aria-labelledby="connect-project">
<h3 id="connect-project" className="text-sm font-medium">
Connect your project
</h3>
<div className="mt-3 overflow-hidden rounded-md border">
<div className="border-b bg-muted/30 px-4">
<LineTabs
value={codingAgent}
options={["Claude Code", "Codex"]}
onChange={(value) => {
setCodingAgent(value);
setCopied(null);
}}
logos={CODING_AGENT_LOGOS}
/>
</div>
<div className="p-4">
<p className="text-sm leading-6 text-muted-foreground">
Run the setup command in your agent’s project. It uses your <code className="text-xs">LITELLM_API_KEY</code>
.
</p>
<Button className="mt-3" onClick={() => void copy()}>
{copied === command ? (
<Check aria-hidden="true" className="size-4" />
) : (
<Copy aria-hidden="true" className="size-4" />
)}
{copied === command ? "Copied" : "Copy setup command"}
</Button>
<details className="mt-3">
<summary className="w-fit cursor-pointer text-xs text-muted-foreground">View command</summary>
<pre className="mt-3 max-h-56 overflow-auto whitespace-pre-wrap rounded-md bg-muted/40 p-3 text-xs leading-5">
<code>{command}</code>
</pre>
</details>
</div>
</div>
</section>
);
}
function ConnectAgent({
accessToken,
onOpenTrace,
@ -615,27 +517,20 @@ function ConnectAgent({
const proxyUrl = getProxyBaseUrl().replace(/\/$/, "");
const [framework, setFramework] = useState(FRAMEWORKS[0].id);
const [installer, setInstaller] = useState<Installer>("pip");
const [codingAgent, setCodingAgent] = useState<CodingAgent>("Claude Code");
const [tracingKey, setTracingKey] = useState<string | null>(null);
const guide = FRAMEWORKS.find((f) => f.id === framework) ?? FRAMEWORKS[0];
const packages = installPackages(guide);
const install = guide.typescript ? `npm install ${packages}` : PY_INSTALL[installer](packages);
const quickstart = guide.quickstart.replace("{PROXY}", proxyUrl);
const install = guide.install?.startsWith("pip install ")
? PY_INSTALL[installer](guide.install.slice("pip install ".length))
: guide.install;
const model = EXAMPLE_MODEL;
const quickstart = frameworkSnippet(guide, proxyUrl, model, !!tracingKey);
return (
<>
{!readOnly && (
<Step title="See it work in one click">
<p className="mb-3 text-sm leading-6 text-muted-foreground">
Send a small sample run (an agent, an LLM call and a tool call) to confirm tracing works end to end.
</p>
<SendTestTrace accessToken={accessToken} onOpenTrace={onOpenTrace} />
</Step>
)}
<Endpoints proxyUrl={proxyUrl} />
<Endpoints proxyUrl={proxyUrl}>
{!readOnly && <SendTestTrace accessToken={accessToken} onOpenTrace={onOpenTrace} />}
</Endpoints>
<div className="mt-6 space-y-2">
<label id="tracing-framework" className="text-sm font-medium">
<label id="tracing-framework" className="block text-sm font-medium">
Your agent framework
</label>
<Select
@ -646,14 +541,14 @@ function ConnectAgent({
>
<SelectTrigger aria-labelledby="tracing-framework" className="w-full">
<SelectValue>
<img src={guide.logo} alt="" className="size-4" />
<img src={guide.logo} alt="" className="size-4 shrink-0 object-contain" />
{guide.label}
</SelectValue>
</SelectTrigger>
<SelectContent>
{FRAMEWORKS.map((f) => (
<SelectItem key={f.id} value={f.id}>
<img src={f.logo} alt="" className="size-4" />
<img src={f.logo} alt="" className="size-4 shrink-0 object-contain" />
{f.label}
</SelectItem>
))}
@ -661,87 +556,91 @@ function ConnectAgent({
</Select>
</div>
<Step title="Get a LiteLLM key">
{canMintTracingKey && !readOnly ? (
<TracingKey accessToken={accessToken} tracingKey={tracingKey} onCreated={setTracingKey} />
) : (
<p className="text-sm text-muted-foreground">
Use any LiteLLM virtual key you already have, or ask a proxy admin for one.
</p>
)}
</Step>
<Step
title={
<span className="inline-flex items-center gap-1.5">
Let <img src={anthropicLogo.src} alt="" className="size-3.5" /> Claude Code or
<img src={openaiLogo.src} alt="" className="size-3.5" /> Codex connect it
</span>
}
>
<p className="mb-3 text-sm leading-6 text-muted-foreground">
Run this in your agent’s project. It starts your coding agent with the setup task and reads the key from
LITELLM_API_KEY.
</p>
<CodeBlock
code={codingAgentCommand(codingAgent, codingAgentPrompt(proxyUrl, guide))}
wrap
tabs={
<LineTabs
value={codingAgent}
options={["Claude Code", "Codex"]}
onChange={setCodingAgent}
logos={CODING_AGENT_LOGOS}
/>
}
/>
</Step>
<details className="mt-6 border-y py-4">
<summary className="cursor-pointer text-sm font-medium">Set up manually</summary>
<Step title="Install dependencies">
<CodeBlock
code={install}
tabs={
guide.typescript ? (
<FileLabel>npm</FileLabel>
) : (
<LineTabs value={installer} options={["pip", "uv"]} onChange={setInstaller} />
)
}
/>
<CodingAgentSetup proxyUrl={proxyUrl} guide={guide} model={model} />
<details className="mt-6 border-t pt-6">
<summary className="w-fit cursor-pointer text-sm font-medium">Set up manually</summary>
<Step title="Get a LiteLLM key">
{canMintTracingKey && !readOnly ? (
<TracingKey accessToken={accessToken} tracingKey={tracingKey} onCreated={setTracingKey} />
) : (
<p className="text-sm text-muted-foreground">
Use any LiteLLM virtual key you already have, or ask a proxy admin for one.
</p>
)}
</Step>
{install && (
<Step title="Install dependencies">
<CodeBlock
code={install}
wrap
tabs={
guide.language === "typescript" ? (
<FileLabel>npm</FileLabel>
) : (
<LineTabs value={installer} options={["pip", "uv"]} onChange={setInstaller} />
)
}
/>
</Step>
)}
{guide.plugin && (
<Step title="Enable the plugin">
<p className="text-sm leading-6 text-muted-foreground">
Install and enable the{" "}
<a className="underline underline-offset-4" href={guide.plugin.url} target="_blank" rel="noreferrer">
{guide.plugin.label}
</a>
. {guide.plugin.instruction}
</p>
</Step>
)}
<Step title="Configure environment">
<p className="mb-3 text-sm leading-6 text-muted-foreground">
{tracingKey ? (
<>
Tracing uses the key above. Set <code>LITELLM_API_KEY</code> to a key with model access.
</>
) : (
<>
Set <code>LITELLM_API_KEY</code> to your LiteLLM key.
</>
)}
</p>
<CodeBlock
code={tracingEnvSnippet(proxyUrl, tracingKey)}
display={tracingEnvSnippet(proxyUrl, tracingKey && maskSecret(tracingKey))}
tabs={<FileLabel>Shell</FileLabel>}
wrap
/>
</Step>
<Step title="Run your agent">
<p className="mb-2 text-sm text-muted-foreground">
Replace the example model with a model configured on your proxy.
<p className="mb-3 text-sm leading-6 text-muted-foreground">
{guide.existingModel ? "Keep your existing model settings. " : "Save and run this example. "}
Replace <code>research_agent</code> with your agent’s name.
{!guide.existingModel && " Use a model configured on this proxy."}
</p>
<CodeBlock
code={guide.typescript ? quickstart : `import os\n\n${quickstart}`}
tabs={<FileLabel>{guide.typescript ? "my_agent.ts" : "my_agent.py"}</FileLabel>}
/>
<div className="mt-3">
<CodeBlock code={guide.typescript ? TS_RUN_SNIPPET : PY_RUN_SNIPPET} tabs={<FileLabel>Shell</FileLabel>} />
</div>
<CodeBlock code={quickstart} tabs={<FileLabel>{guide.fileName}</FileLabel>} wrap />
{guide.note && <p className="mt-3 text-sm leading-6 text-muted-foreground">{guide.note}</p>}
{!guide.existingModel && (
<div className="mt-3">
<CodeBlock
code={guide.language === "typescript" ? "npx tsx agent.mts" : "python agent.py"}
tabs={<FileLabel>Shell</FileLabel>}
/>
</div>
)}
<a
href={`${DOCS_URL}?framework=${guide.id}#send-your-first-trace`}
className="mt-3 inline-flex items-center gap-1 text-sm underline underline-offset-4"
target="_blank"
rel="noreferrer"
>
View in docs <ArrowUpRight aria-hidden="true" className="size-3.5" />
</a>
</Step>
</details>
<TraceReceipt connected={connected} checked={checked} checking={checking} onCheck={onCheck} />
{!connected && (
<section className="mt-6">
<h3 className="mb-2 text-sm font-medium">
Example run <span className="font-normal text-muted-foreground">(sample data, not your runs)</span>
</h3>
<TracePreview />
</section>
)}
</>
);
}
@ -755,6 +654,7 @@ export function TracingSetupCard({
checking = false,
readOnly = false,
canMintTracingKey = false,
onDemo,
}: {
detail: string | null;
accessToken: string;
@ -764,6 +664,7 @@ export function TracingSetupCard({
checking?: boolean;
readOnly?: boolean;
canMintTracingKey?: boolean;
onDemo?: () => void;
}) {
const [checked, setChecked] = useState(false);
const enabled = detail === null;
@ -773,9 +674,18 @@ export function TracingSetupCard({
};
return (
<div className="w-full py-6 pb-12" data-testid="tracing-setup-card">
<div className="flex items-start gap-4">
<div className="w-full max-w-3xl pb-8" data-testid="tracing-setup-card">
{onDemo && <LensPreviewButton onClick={onDemo} />}
<div className="flex flex-wrap items-center gap-x-4 gap-y-2">
<h2 className="text-xl font-semibold tracking-tight">{setupTitle(enabled, connected)}</h2>
<span role="status" className="inline-flex items-center gap-1.5 text-xs text-muted-foreground">
{enabled ? (
<ActiveDot />
) : (
<span aria-hidden="true" className="size-1.5 rounded-full bg-muted-foreground/50" />
)}
{enabled ? "Tracing enabled" : "Tracing is not enabled"}
</span>
<a
className="ml-auto inline-flex shrink-0 items-center gap-1 text-sm underline underline-offset-4"
href={DOCS_URL}
@ -790,10 +700,6 @@ export function TracingSetupCard({
? "Send your agent’s runs to LiteLLM to see its inputs, outputs, and tool calls."
: "Tracing needs ClickHouse and a small update to your LiteLLM proxy configuration."}
</p>
<p role="status" className="mt-4 flex items-center gap-2 text-sm">
{enabled && <ActiveDot />}
{enabled ? "Tracing enabled" : "Tracing is not enabled"}
</p>
{enabled ? (
<ConnectAgent
accessToken={accessToken}

4
ui/litellm-dashboard/src/components/view_logs/TraceView/spanProvider.ts
View file

@ -1,3 +1,4 @@
import { useLensDemo } from "@/components/lens/LensDemoContext";
import { useModelCostMap } from "@/app/(dashboard)/hooks/models/useModelCostMap";
import { getProviderLogoAndName } from "@/components/provider_info_helpers";
@ -29,6 +30,7 @@ export function resolveSpanProvider(model: string | null, lookup: ProviderLookup
}
export function useSpanProvider(model: string | null): string | null {
const { data } = useModelCostMap();
const demo = useLensDemo();
const { data } = useModelCostMap(!demo && model !== null);
return resolveSpanProvider(model, costMapLookup(data));
}

501
ui/litellm-dashboard/src/components/view_logs/TraceView/tracingSetupGuides.ts Normal file
View file

@ -0,0 +1,501 @@
import langgraphLogo from "../../../../public/assets/logos/langgraph-color.svg";
import langchainLogo from "../../../../public/assets/logos/langchain.svg";
import openaiAgentsLogo from "../../../../public/assets/logos/openai-agents.svg";
import anthropicLogo from "../../../../public/assets/logos/anthropic.svg";
import crewaiLogo from "../../../../public/assets/logos/crewai-color.svg";
import pydanticAiLogo from "../../../../public/assets/logos/pydantic-ai-color.svg";
import llamaindexLogo from "../../../../public/assets/logos/llamaindex-color.svg";
import vercelLogo from "../../../../public/assets/logos/vercel.svg";
import otelLogo from "../../../../public/assets/logos/opentelemetry.svg";
import adkLogo from "../../../../public/assets/logos/google-adk.png";
import strandsLogo from "../../../../public/assets/logos/strands.svg";
import hermesLogo from "../../../../public/assets/logos/hermes.png";
import openclawLogo from "../../../../public/assets/logos/openclaw.png";
export interface FrameworkGuide {
id: string;
label: string;
logo: string;
install?: string;
language: "python" | "typescript" | "json" | "yaml";
quickstart: string;
fileName: string;
existingModel?: boolean;
note?: string;
plugin?: { label: string; url: string; instruction: string };
}
export const FRAMEWORKS: readonly FrameworkGuide[] = [
{
id: "deepagents",
label: "DeepAgents",
logo: langgraphLogo.src,
install:
"pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http deepagents openinference-instrumentation-langchain langchain-openai",
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from deepagents import create_deep_agent
from langchain_openai import ChatOpenAI
model = ChatOpenAI(
model={MODEL},
base_url={BASE_URL},
api_key=os.environ["LITELLM_API_KEY"],
)
agent = create_deep_agent(name=AGENT_NAME, model=model, tools=[])
result = agent.invoke({"messages": [{"role": "user", "content": "What is an agent trace?"}]})
print(result["messages"][-1].content)`,
fileName: "agent.py",
},
{
id: "langgraph",
label: "LangGraph",
logo: langgraphLogo.src,
install:
"pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http langgraph openinference-instrumentation-langchain langchain-openai",
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from langgraph.graph import END, START, MessagesState, StateGraph
from langchain_openai import ChatOpenAI
model = ChatOpenAI(
model={MODEL},
base_url={BASE_URL},
api_key=os.environ["LITELLM_API_KEY"],
)
graph = StateGraph(MessagesState)
graph.add_node("answer", lambda state: {"messages": [model.invoke(state["messages"])]})
graph.add_edge(START, "answer")
graph.add_edge("answer", END)
agent = graph.compile(name=AGENT_NAME)
result = agent.invoke({"messages": [{"role": "user", "content": "What is an agent trace?"}]})
print(result["messages"][-1].content)`,
fileName: "agent.py",
},
{
id: "langchain",
label: "LangChain",
logo: langchainLogo.src,
install:
"pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http langchain openinference-instrumentation-langchain langchain-openai",
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
model = ChatOpenAI(
model={MODEL},
base_url={BASE_URL},
api_key=os.environ["LITELLM_API_KEY"],
)
agent = create_agent(name=AGENT_NAME, model=model, tools=[])
result = agent.invoke({"messages": [{"role": "user", "content": "What is an agent trace?"}]})
print(result["messages"][-1].content)`,
fileName: "agent.py",
},
{
id: "openai-agents",
label: "OpenAI Agents",
logo: openaiAgentsLogo.src,
install:
"pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http openai-agents openinference-instrumentation-openai-agents",
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from agents import Agent, Runner
from agents import OpenAIChatCompletionsModel
from openai import AsyncOpenAI
client = AsyncOpenAI(base_url={BASE_URL}, api_key=os.environ["LITELLM_API_KEY"])
model = OpenAIChatCompletionsModel(model={MODEL}, openai_client=client)
agent = Agent(name=AGENT_NAME, model=model)
result = Runner.run_sync(agent, "What is an agent trace?")
print(result.final_output)`,
fileName: "agent.py",
},
{
id: "claude",
label: "Claude Agent SDK",
logo: anthropicLogo.src,
install:
"pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http claude-agent-sdk openinference-instrumentation-claude-agent-sdk",
language: "python",
quickstart: `import asyncio
import os
AGENT_NAME = "research_agent"
os.environ["OTEL_RESOURCE_ATTRIBUTES"] = f"gen_ai.agent.name={AGENT_NAME}"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from claude_agent_sdk import ClaudeAgentOptions, ResultMessage, query
options = ClaudeAgentOptions(
model={MODEL},
env={"ANTHROPIC_BASE_URL": {PROXY_URL}, "ANTHROPIC_AUTH_TOKEN": os.environ["LITELLM_API_KEY"]},
tools=[],
max_turns=1,
)
async def main():
async for message in query(prompt="What is an agent trace?", options=options):
if isinstance(message, ResultMessage):
print(message.result)
asyncio.run(main())`,
fileName: "agent.py",
note: "This captures SDK input and output; internal model calls are not exposed by this instrumentor.",
},
{
id: "crewai",
label: "CrewAI",
logo: crewaiLogo.src,
install:
"pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http crewai openinference-instrumentation-crewai",
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from crewai import Agent, Crew, Task
from crewai import LLM
model = LLM(
model={OPENAI_MODEL},
base_url={BASE_URL},
api_key=os.environ["LITELLM_API_KEY"],
)
agent = Agent(
role=AGENT_NAME,
goal="Answer questions clearly",
backstory="You explain technical concepts.",
llm=model,
)
task = Task(description="What is an agent trace?", expected_output="A short answer", agent=agent)
print(Crew(agents=[agent], tasks=[task]).kickoff())`,
fileName: "agent.py",
},
{
id: "pydantic-ai",
label: "Pydantic AI",
logo: pydanticAiLogo.src,
install: "pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http pydantic-ai",
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
model = OpenAIChatModel(
{MODEL},
provider=OpenAIProvider(base_url={BASE_URL}, api_key=os.environ["LITELLM_API_KEY"]),
)
Agent.instrument_all()
agent = Agent(model, name=AGENT_NAME)
print(agent.run_sync("What is an agent trace?").output)`,
fileName: "agent.py",
},
{
id: "llamaindex",
label: "LlamaIndex",
logo: llamaindexLogo.src,
install:
"pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http llama-index-core openinference-instrumentation-llama-index llama-index-llms-openai-like",
language: "python",
quickstart: `import asyncio
import os
AGENT_NAME = "research_agent"
os.environ["OTEL_RESOURCE_ATTRIBUTES"] = f"gen_ai.agent.name={AGENT_NAME}"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
LlamaIndexInstrumentor().instrument()
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike
model = OpenAILike(
model={MODEL},
api_base={BASE_URL},
api_key=os.environ["LITELLM_API_KEY"],
is_chat_model=True,
is_function_calling_model=True,
)
agent = FunctionAgent(name=AGENT_NAME, llm=model, tools=[])
async def main():
result = await agent.run(user_msg="What is an agent trace?")
print(result)
asyncio.run(main())`,
fileName: "agent.py",
note: "The resource attribute supplies the agent name because this instrumentor does not export FunctionAgent.name.",
},
{
id: "adk",
label: "Google ADK",
logo: adkLogo.src,
install: "pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http google-adk",
language: "python",
quickstart: `import asyncio
import os
AGENT_NAME = "research_agent"
os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "SPAN_ONLY"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from google.adk.agents import Agent
from google.adk.runners import InMemoryRunner
from google.adk.models.lite_llm import LiteLlm
model = LiteLlm(
model={OPENAI_MODEL},
api_base={BASE_URL},
api_key=os.environ["LITELLM_API_KEY"],
)
agent = Agent(name=AGENT_NAME, model=model)
asyncio.run(InMemoryRunner(agent=agent).run_debug("What is an agent trace?"))`,
fileName: "agent.py",
note: "SPAN_ONLY records the messages needed to inspect and investigate the run.",
},
{
id: "strands",
label: "Strands",
logo: strandsLogo.src,
install: 'pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http "strands-agents[otel]" openai',
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
os.environ["OTEL_SEMCONV_STABILITY_OPT_IN"] = "gen_ai_latest_experimental,gen_ai_span_attributes_only"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from strands import Agent
from strands.models.openai import OpenAIModel
model = OpenAIModel(
client_args={"base_url": {BASE_URL}, "api_key": os.environ["LITELLM_API_KEY"]},
model_id={MODEL},
)
agent = Agent(name=AGENT_NAME, model=model)
print(agent("What is an agent trace?"))`,
fileName: "agent.py",
note: "The semantic-convention setting enables message content in spans.",
},
{
id: "vercel",
label: "Vercel AI SDK",
logo: vercelLogo.src,
install:
"npm install ai @ai-sdk/otel @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http @ai-sdk/openai-compatible",
language: "typescript",
quickstart: `import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { OpenTelemetry } from "@ai-sdk/otel";
import { generateText, registerTelemetry } from "ai";
const sdk = new NodeSDK({ traceExporter: new OTLPTraceExporter() });
sdk.start();
registerTelemetry(new OpenTelemetry());
const AGENT_NAME = "research_agent";
const litellm = createOpenAICompatible({
name: "litellm",
baseURL: {BASE_URL},
apiKey: process.env.LITELLM_API_KEY,
});
const model = litellm({MODEL});
try {
const { text } = await generateText({
model,
prompt: "What is an agent trace?",
telemetry: { functionId: AGENT_NAME },
});
console.log(text);
} finally {
await sdk.shutdown();
}`,
fileName: "agent.mts",
},
{
id: "openclaw",
label: "OpenClaw",
logo: openclawLogo.src,
language: "json",
quickstart: `{
"agents": {
"list": [{ "id": "research_agent" }]
},
"plugins": {
"entries": { "diagnostics-otel": { "enabled": true } }
},
"diagnostics": {
"enabled": true,
"otel": {
"enabled": true,
"tracesEndpoint": "{TRACE_URL}",
"headers": { "Authorization": "Bearer \${LITELLM_API_KEY}" },
"captureContent": true,
"traces": true,
"metrics": false,
"logs": false,
"sampleRate": 1
}
}
}`,
existingModel: true,
fileName: "openclaw.json",
note: 'Set LITELLM_API_KEY to your LiteLLM key, then run openclaw agent --local --session-id first-trace --message "What is an agent trace?". Select research_agent in Lens. Restart an existing gateway after changing the config.',
plugin: {
label: "diagnostics-otel plugin",
url: "https://docs.openclaw.ai/plugins/reference/diagnostics-otel",
instruction:
"Set your agent ID once in ~/.openclaw/openclaw.json. Keep your existing model and workspace settings when adding the tracing configuration.",
},
},
{
id: "hermes",
label: "Hermes",
logo: hermesLogo.src,
language: "yaml",
quickstart: `resource_attributes:
gen_ai.agent.name: research_agent
content_capture: full
backends:
- type: otlp
endpoint: {TRACE_URL}
headers:
Authorization: "Bearer \${LITELLM_API_KEY}"
metrics: false
logs: false`,
existingModel: true,
fileName: "hermes_otel.yaml",
note: "Start a new Hermes session and ask a question. The configured name research_agent appears in Lens. Hermes' built-in diagnostic telemetry alone does not include the conversation content needed for investigations.",
plugin: {
label: "community hermes-otel plugin",
url: "https://github.com/briancaffey/hermes-otel#install",
instruction: "Set LITELLM_API_KEY to your LiteLLM key, then add this to ~/.hermes/hermes_otel.yaml.",
},
},
{
id: "otel",
label: "OpenTelemetry",
logo: otelLogo.src,
install: "pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http openai",
language: "python",
quickstart: `import os
AGENT_NAME = "research_agent"
from opentelemetry.instrumentation.auto_instrumentation import initialize
initialize()
from opentelemetry import trace
from openai import OpenAI
client = OpenAI(base_url={BASE_URL}, api_key=os.environ["LITELLM_API_KEY"])
with trace.get_tracer(__name__).start_as_current_span(AGENT_NAME) as span:
span.set_attribute("gen_ai.agent.name", AGENT_NAME)
span.set_attribute("openinference.span.kind", "AGENT")
span.set_attribute("input.value", "What is an agent trace?")
result = client.chat.completions.create(
model={MODEL},
messages=[{"role": "user", "content": "What is an agent trace?"}],
)
answer = result.choices[0].message.content
span.set_attribute("output.value", str(answer))
print(answer)`,
fileName: "agent.py",
},
];
export function frameworkSnippet(guide: FrameworkGuide, proxyUrl: string, model: string, tracingKey = false): string {
const values: Record<string, string> = {
MODEL: JSON.stringify(model),
OPENAI_MODEL: JSON.stringify(`openai/${model}`),
BASE_URL: JSON.stringify(`${proxyUrl}/v1`),
PROXY_URL: JSON.stringify(proxyUrl),
TRACE_URL: `${proxyUrl}/v1/traces`,
};
const code = guide.quickstart.replace(
/\{(MODEL|OPENAI_MODEL|BASE_URL|PROXY_URL|TRACE_URL)\}/g,
(_, name: string) => values[name],
);
return tracingKey && guide.existingModel ? code.replaceAll("${LITELLM_API_KEY}", "${LITELLM_TRACING_KEY}") : code;
}

12
ui/litellm-dashboard/src/components/view_logs/TraceView/useAgentTraces.ts
View file

@ -1,3 +1,4 @@
import { useLensDemo } from "@/components/lens/LensDemoContext";
import { useInfiniteQuery, useQuery, type UseQueryOptions } from "@tanstack/react-query";
import moment from "moment";
import { useMemo } from "react";
@ -65,6 +66,7 @@ export function useAgentTraces({
isLiveTail,
enabled,
}: UseAgentTracesOptions): AgentTracesResult {
const demo = useLensDemo();
const fetchPage = (pageParam: unknown): Promise<TracePage> => {
const nowMs = Date.now();
const listOptions: Parameters<typeof agentTraceListCall>[0] = {
@ -73,7 +75,11 @@ export function useAgentTraces({
endMs: isCustomDate ? moment(endTime).valueOf() : nowMs,
cursor: pageParam as string | null,
};
return agentTraceListCall(listOptions);
return demo
? demo.client.get<TracePage>("/v1/traces", {
query: { start_ms: listOptions.startMs, end_ms: listOptions.endMs, cursor: listOptions.cursor },
})
: agentTraceListCall(listOptions);
};
const queryOptions: Parameters<typeof useInfiniteQuery<TracePage, Error>>[0] = {
queryKey: ["agentTraces", accessToken, startTime, endTime, isCustomDate],
@ -106,9 +112,11 @@ export function useAgentTraces({
}
export function useTraceAvailability(accessToken: string, enabled: boolean) {
const demo = useLensDemo();
const client = demo?.client ?? apiClient;
const options: UseQueryOptions<TracePage, Error, boolean> = {
queryKey: ["trace-availability", accessToken],
queryFn: () => apiClient.get<TracePage>("/v1/traces", { accessToken, query: { start_ms: 0 } }),
queryFn: () => client.get<TracePage>("/v1/traces", { accessToken, query: { start_ms: 0 } }),
select: (page: TracePage) => page.data.length > 0,
enabled,
retry: false,

105
ui/litellm-dashboard/src/components/view_logs/index.tsx
View file

@ -1,105 +0,0 @@
import { useState } from "react";
import useCan from "@/app/(dashboard)/hooks/useCan";
import DeletedKeysPage from "../DeletedKeysPage/DeletedKeysPage";
import DeletedTeamsPage from "../DeletedTeamsPage/DeletedTeamsPage";
import AuditLogsPanel from "./AuditLogsPanel";
import RequestLogsPanel from "./RequestLogsPanel";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { UiLoadingSpinner } from "@/components/ui/ui-loading-spinner";
interface SpendLogsTableProps {
accessToken: string | null;
token: string | null;
userRole: string | null;
userID: string | null;
premiumUser: boolean;
}
type LogsTabId = "request logs" | "audit logs" | "deleted keys" | "deleted teams";
interface LogsTab {
id: LogsTabId;
label: string;
}
const REQUEST_LOGS_TAB: LogsTab = { id: "request logs", label: "Request Logs" };
const AUDIT_LOGS_TAB: LogsTab = { id: "audit logs", label: "Audit Logs" };
const DELETED_KEYS_TAB: LogsTab = { id: "deleted keys", label: "Deleted Keys" };
const DELETED_TEAMS_TAB: LogsTab = { id: "deleted teams", label: "Deleted Teams" };
const tabContentClassName = (tabId: LogsTabId): string =>
tabId === REQUEST_LOGS_TAB.id ? "flex min-h-0 flex-1 flex-col" : "min-h-0 flex-1 overflow-y-auto";
export default function SpendLogsTable({ accessToken, token, userRole, userID, premiumUser }: SpendLogsTableProps) {
const [activeTab, setActiveTab] = useState<LogsTabId>(REQUEST_LOGS_TAB.id);
const canViewAuditLogs = useCan("viewAuditLogs");
const canViewDeletedTeams = useCan("viewDeletedTeams");
if (!accessToken || !token || !userRole || !userID) {
return (
<div role="status" aria-busy="true" aria-label="Loading" className="flex h-64 items-center justify-center">
<UiLoadingSpinner className="size-8 text-primary" />
</div>
);
}
const tabs: LogsTab[] = [
REQUEST_LOGS_TAB,
...(canViewAuditLogs ? [AUDIT_LOGS_TAB] : []),
DELETED_KEYS_TAB,
...(canViewDeletedTeams ? [DELETED_TEAMS_TAB] : []),
];
const renderPanel = (tabId: LogsTabId) => {
switch (tabId) {
case "request logs":
return (
<RequestLogsPanel
accessToken={accessToken}
token={token}
userRole={userRole}
userID={userID}
isActive={activeTab === "request logs"}
/>
);
case "audit logs":
return (
<AuditLogsPanel
userID={userID}
userRole={userRole}
token={token}
accessToken={accessToken}
isActive={activeTab === "audit logs"}
premiumUser={premiumUser}
/>
);
case "deleted keys":
return <DeletedKeysPage />;
case "deleted teams":
return <DeletedTeamsPage />;
}
};
return (
<div className="flex h-full w-full flex-col px-4 pb-4">
<Tabs
value={activeTab}
onValueChange={(value) => setActiveTab(value as LogsTabId)}
className="min-h-0 flex-1 gap-0"
>
<TabsList variant="line" className="-mx-4 h-10 w-auto justify-start gap-4 border-b border-border px-4">
{tabs.map((tab) => (
<TabsTrigger key={tab.id} value={tab.id} className="flex-none gap-1.5 px-0 text-[13px] font-normal">
{tab.label}
</TabsTrigger>
))}
</TabsList>
{tabs.map((tab) => (
<TabsContent key={tab.id} value={tab.id} keepMounted className={tabContentClassName(tab.id)}>
{renderPanel(tab.id)}
</TabsContent>
))}
</Tabs>
</div>
);
}