LiteLLM Proxy · Latency Diagnostics
+Issues Detected
+ {issues.length === 0 && latency.sample_count > 0 && ( + ++ {latency.sample_count === 0 ? "Send traffic to start analysis." : "No issues found — proxy looks healthy."} +
+ ) : ( +| + | Issue | +Suggested Fix | +
|---|---|---|
| + + | +
+ {issue.title} +{issue.description} + |
+
+ {issue.fix_snippet ? (
+
+ {issue.fix_snippet}
+
+ {issue.fix} + )} + |
+
Overhead Over Time
+LiteLLM-added latency · last 10 min · 10s resolution
+ + {/* Plain-English summary */} + {latency.sample_count > 0 && latency.overhead_histogram && (() => { + const hist = latency.overhead_histogram; + const total = hist.reduce((s, b) => s + b.count, 0); + const under50 = hist.filter(b => ["0-5","5-10","10-25","25-50"].includes(b.bucket)).reduce((s,b) => s+b.count, 0); + const pct = total > 0 ? Math.round(under50 / total * 100) : 0; + const worstBucket = [...hist].reverse().find(b => b.count > 0); + return ( +{total}
+total requests
+= 90 ? "text-green-600" : pct >= 70 ? "text-amber-500" : "text-red-500"}`}>{pct}%
+under 50ms overhead
+{latency.overhead?.p95_ms ?? "—"}ms
+p95 overhead
+{worstBucket.count}
+requests > {worstBucket.bucket.split("-")[0]}ms
+WORKER PROVISIONING
+ {workersLow &&CPU Cores
+{workers.cpu_count}
+Workers
++ {workers.num_workers} +
+CPU Usage
+80 ? "text-red-500" : workers.cpu_percent > 60 ? "text-orange-500" : "text-green-600"}`}> + {workers.cpu_percent}% +
+Recommended: {2 * workers.cpu_count + 1} workers (2× CPU + 1)
+ )} +CONNECTIONS
+Database
+ + {connection_pools.db.connected ? "● Connected" : "● Disconnected"} + +Pool limit
+{connection_pools.db.pool_limit}
+Redis
+ + {connection_pools.redis.enabled ? "● Connected" : "● Not configured"} + +Max connections
+{connection_pools.redis.max_connections ?? "∞"}
+In-Flight Requests
+ + {connection_pools.in_flight_requests ?? "—"} + asyncio tasks + ++ Concurrent asyncio tasks (proxy-wide) · last 10 min +
+HTTP Client Pool Utilization
+ + {connection_pools.http.aiohttp_active ?? "—"} + + {" "}/ {connection_pools.http.aiohttp_limit} limit + {connection_pools.http.aiohttp_pct != null && ` · ${connection_pools.http.aiohttp_pct}%`} + + ++ aiohttp active connections ÷ pool limit · amber line = 80% · last 10 min +
+Per-Model Overhead
+Sorted by overhead avg · last {latency.sample_count} requests
+| Model | +Overhead avg | +Overhead p95 | +LLM API avg | +Total avg | +Requests | +
|---|---|---|---|---|---|
| + {row.model} + | ++ 20 ? "text-orange-500 font-semibold" : "text-gray-700"}> + {row.overhead ? `${row.overhead.avg_ms}ms` : "—"} + + {overheadPct != null && ( + ({overheadPct}%) + )} + | ++ {row.overhead ? `${row.overhead.p95_ms}ms` : "—"} + | ++ {row.llm_api ? `${row.llm_api.avg_ms}ms` : "—"} + | ++ {row.total ? `${row.total.avg_ms}ms` : "—"} + | +{row.sample_count} | +
Configuration Summary
+Log Level
+Detailed Timing
+Workers / CPU
+ {workers.num_workers} / {workers.cpu_count} +Sample Count
+ {latency.sample_count} +