scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

docs: add proxy req/s

Browse source
This commit is contained in:
Krrish Dholakia 2023-12-08 07:33:33 -08:00
parent 0406e01c6d
commit d548836903
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/my-website/docs/proxy/deploy.md
View file

@ -76,6 +76,8 @@ Your LiteLLM container should be running now on the defined port e.g. `8000`.
## LiteLLM Proxy Performance
LiteLLM proxy has been load tested to handle 1500 req/s.
### Throughput - 30% Increase
LiteLLM proxy + Load Balancer gives **30% increase** in throughput compared to Raw OpenAI API
<Image img={require('../img/throughput.png')} />