scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-13 08:52:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 175

update benchmarks

Browse source
This commit is contained in:
AlexsanderHamir 2025-10-11 18:06:37 -07:00
parent 6df051e87b
commit 2e57d19a55
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/my-website/docs/benchmarks.md
View file

@ -44,8 +44,8 @@ In these tests the baseline latency characteristics are measured against a fake-
| | Aggregated | 77 | 130 | 180 | 57.53 | 2340 |
#### Key Findings
- Doubling from 2 to 4 LiteLLM instances cuts median latency from 200 ms to 100 ms.
- High-percentile latencies drop significantly: P95 from 430 ms to 130 ms, P99 from 930 ms to 180 ms.
- Doubling from 2 to 4 LiteLLM instances halves median latency: 200 ms → 100 ms.
- High-percentile latencies drop significantly: P95 630 ms → 150 ms, P99 1,200 ms → 240 ms.
- Setting workers equal to CPU count gives optimal performance.
## Machine Spec used for testing