From 0292b84dc47473ddeff29bd5a86f529bc523034b Mon Sep 17 00:00:00 2001 From: Alexsander Hamir Date: Sat, 15 Nov 2025 20:12:32 -0800 Subject: [PATCH] docs: add missing details to benchmark comparison (#16690) --- docs/my-website/docs/benchmarks.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/my-website/docs/benchmarks.md b/docs/my-website/docs/benchmarks.md index f00732450d1..4e4234949f8 100644 --- a/docs/my-website/docs/benchmarks.md +++ b/docs/my-website/docs/benchmarks.md @@ -125,18 +125,23 @@ class MyUser(HttpUser): ## LiteLLM vs Portkey Performance Comparison **Test Configuration**: 4 CPUs, 8 GB RAM per instance | Load: 1k concurrent users, 500 ramp-up +**Versions:** Portkey **v1.14.0** | LiteLLM **v1.79.1-stable** +**Test Duration:** 5 minutes ### Multi-Instance (4×) Performance -| Metric | Portkey (no DB) | LiteLLM (with DB) | -| ------------------- | --------------- | ----------------- | -| **Total Requests** | 293,796 | 312,405 | -| **Failed Requests** | 0 | 0 | -| **Median Latency** | 100 ms | 100 ms | -| **p95 Latency** | 230 ms | 150 ms | -| **p99 Latency** | 500 ms | 240 ms | -| **Average Latency** | 123 ms | 111 ms | -| **Current RPS** | 1,170.9 | 1,170 | +| Metric | Portkey (no DB) | LiteLLM (with DB) | Comment | +| ------------------- | --------------- | ----------------- | -------------- | +| **Total Requests** | 293,796 | 312,405 | LiteLLM higher | +| **Failed Requests** | 0 | 0 | Same | +| **Median Latency** | 100 ms | 100 ms | Same | +| **p95 Latency** | 230 ms | 150 ms | LiteLLM lower | +| **p99 Latency** | 500 ms | 240 ms | LiteLLM lower | +| **Average Latency** | 123 ms | 111 ms | LiteLLM lower | +| **Current RPS** | 1,170.9 | 1,170 | Same | + + +*Lower is better for latency metrics; higher is better for requests and RPS.* ### Technical Insights