From 0406e01c6db1f9043b78ab4dc5bc09506f0c7a63 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Fri, 8 Dec 2023 07:32:23 -0800 Subject: [PATCH] docs(deploy.md): add proxy perf to docs --- docs/my-website/docs/proxy/deploy.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/my-website/docs/proxy/deploy.md b/docs/my-website/docs/proxy/deploy.md index 5c4657ea37d..745d7996649 100644 --- a/docs/my-website/docs/proxy/deploy.md +++ b/docs/my-website/docs/proxy/deploy.md @@ -73,3 +73,13 @@ Your LiteLLM container should be running now on the defined port e.g. `8000`. + +## LiteLLM Proxy Performance + +### Throughput - 30% Increase +LiteLLM proxy + Load Balancer gives **30% increase** in throughput compared to Raw OpenAI API + + +### Latency Added - 0.00325 seconds +LiteLLM proxy adds **0.00325 seconds** latency as compared to using the Raw OpenAI API +