From 277b56438509caf325ae68a0fe18e76ab7eec216 Mon Sep 17 00:00:00 2001
From: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Date: Sat, 8 Nov 2025 16:42:58 -0800
Subject: [PATCH] add: performance improvements to release notes (#16401)

* add: performance improvements to release notes

* fix: mention which endpoint is getting faster

* fix: clarify that reported latency is end-to-end
---
 .../release_notes/v1.79.1-stable/index.md     | 56 +++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/docs/my-website/release_notes/v1.79.1-stable/index.md b/docs/my-website/release_notes/v1.79.1-stable/index.md
index 7b3b89b29b6..d119d3e4ffc 100644
--- a/docs/my-website/release_notes/v1.79.1-stable/index.md
+++ b/docs/my-website/release_notes/v1.79.1-stable/index.md
@@ -54,6 +54,62 @@ pip install litellm==1.80.0
 
 ---
 
+### Performance – `/responses` 19× Lower Median, `/embeddings` 13× Lower p95 Latency
+
+This update fixes two major issues. First, embeddings now go through the same optimized pipeline as chat/completions, benefiting from all performance improvements applied so far. Second, while the responses API was already using the same pipeline, it now also includes our internal network management for connection handling, drastically improving latency and reducing memory overhead from per-request setup.
+
+As a result, end-to-end latency improved across all percentiles:
+
+`/embeddings`
+
+**Median latency:** 290 ms → **230 ms** (−20.7%)
+
+**p95 latency:** 5,700 ms → **430 ms** (−92.5%, ~13× faster)
+
+**p99 latency:** 7,200 ms → **780 ms** (−89.2%)
+
+**Average latency:** 844 ms → **263 ms** (−68.9%)
+
+<br/>
+
+`/responses`
+
+**Median latency:** 3,600 ms → **190 ms** (−94.7%, ~19× faster)
+
+**p95 latency:** 4,300 ms → **280 ms** (−93.5%)
+
+**p99 latency:** 4,600 ms → **590 ms** (−87.2%)
+
+**Average latency:** 3,571 ms → **208 ms** (−94.2%)
+
+**RPS:** 231.3 → **1,058.6** (+357.5%)
+
+### **Test Setup**
+
+**Locust**
+
+- **Concurrent users:** 1,000
+- **Ramp-up:** 500
+
+**System Specs**
+
+- **CPU:** 4 vCPUs
+- **Memory:** 8 GB RAM
+- **LiteLLM Workers:** 4
+- **Instances**: 4
+
+> Database used; Redis unused.
+
+**Configuration (config.yaml)**
+
+View the complete configuration: [gist.github.com/AlexsanderHamir/config.yaml](https://gist.github.com/AlexsanderHamir/550791675fd752befcac6a9e44024652)
+
+**Load Script (no_cache_hits.py)**
+
+View the complete load testing script: [gist.github.com/AlexsanderHamir/no_cache_hits.py](https://gist.github.com/AlexsanderHamir/99d673bf74cdd81fd39f59fa9048f2e8)
+
+---
+
 ## Dependency Upgrades
 
 - **Dependencies**