scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

docs(deploy.md): add proxy perf to docs

Browse source
This commit is contained in:
Krrish Dholakia 2023-12-08 07:32:23 -08:00
parent 2f1c5aa0c7
commit 0406e01c6d
1 changed files with 10 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
docs/my-website/docs/proxy/deploy.md
View file

@ -73,3 +73,13 @@ Your LiteLLM container should be running now on the defined port e.g. `8000`.
<iframe width="840" height="500" src="https://www.loom.com/embed/805964b3c8384b41be180a61442389a3" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
## LiteLLM Proxy Performance
### Throughput - 30% Increase
LiteLLM proxy + Load Balancer gives **30% increase** in throughput compared to Raw OpenAI API
<Image img={require('../img/throughput.png')} />
### Latency Added - 0.00325 seconds
LiteLLM proxy adds **0.00325 seconds** latency as compared to using the Raw OpenAI API
<Image img={require('../img/latency.png')} />