scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

docs: add missing details to benchmark comparison (#16690)

Browse source
This commit is contained in:
Alexsander Hamir 2025-11-15 20:12:32 -08:00 • committed by GitHub
parent 4be0c6a226
commit 0292b84dc4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 14 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
docs/my-website/docs/benchmarks.md
View file

@ -125,18 +125,23 @@ class MyUser(HttpUser):
## LiteLLM vs Portkey Performance Comparison
**Test Configuration**: 4 CPUs, 8 GB RAM per instance | Load: 1k concurrent users, 500 ramp-up
**Versions:** Portkey **v1.14.0** | LiteLLM **v1.79.1-stable**
**Test Duration:** 5 minutes
### Multi-Instance (4×) Performance
| Metric | Portkey (no DB) | LiteLLM (with DB) |
| ------------------- | --------------- | ----------------- |
| **Total Requests** | 293,796 | 312,405 |
| **Failed Requests** | 0 | 0 |
| **Median Latency** | 100 ms | 100 ms |
| **p95 Latency** | 230 ms | 150 ms |
| **p99 Latency** | 500 ms | 240 ms |
| **Average Latency** | 123 ms | 111 ms |
| **Current RPS** | 1,170.9 | 1,170 |
| Metric | Portkey (no DB) | LiteLLM (with DB) | Comment |
| ------------------- | --------------- | ----------------- | -------------- |
| **Total Requests** | 293,796 | 312,405 | LiteLLM higher |
| **Failed Requests** | 0 | 0 | Same |
| **Median Latency** | 100 ms | 100 ms | Same |
| **p95 Latency** | 230 ms | 150 ms | LiteLLM lower |
| **p99 Latency** | 500 ms | 240 ms | LiteLLM lower |
| **Average Latency** | 123 ms | 111 ms | LiteLLM lower |
| **Current RPS** | 1,170.9 | 1,170 | Same |
*Lower is better for latency metrics; higher is better for requests and RPS.*
### Technical Insights