scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 85

docs: tweak benchmarks wording

Browse source
This commit is contained in:
Ryan Crabbe 2026-02-23 09:50:02 -08:00
parent 5b41b009f6
commit 8244ad1f0e
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/my-website/docs/benchmarks.md
View file

@ -7,7 +7,7 @@ Benchmarks for LiteLLM Gateway (Proxy Server) tested against a fake OpenAI endpo
## Setting Up Benchmarking with Network Mock
The fastest way to benchmark proxy overhead is using `network_mock` mode. This intercepts outbound requests at the httpx transport layer and returns canned responses — no external endpoint needed.
The fastest way to benchmark proxy overhead is using `network_mock` mode. This intercepts outbound requests at the httpx transport layer and returns canned responses, no need for setting up a mock provider.
**1. Create a proxy config:**
@ -41,7 +41,7 @@ litellm --config benchmark_config.yaml --port 4000 --num_workers 8
python scripts/benchmark_mock.py --requests 2000 --max-concurrent 200 --runs 3
```
This measures pure proxy overhead (auth, routing, logging) without any network latency to a real or fake provider.
This measures pure proxy overhead on the hot path without any network latency to a real or fake provider.
## Setting Up a Fake OpenAI Endpoint