diff --git a/docs/my-website/img/release_notes/perf_77_5.png b/docs/my-website/img/release_notes/perf_77_5.png new file mode 100644 index 00000000000..3aaebaf6164 Binary files /dev/null and b/docs/my-website/img/release_notes/perf_77_5.png differ diff --git a/docs/my-website/release_notes/v1.77.5-stable/index.md b/docs/my-website/release_notes/v1.77.5-stable/index.md index a321060d306..9d5df8ac260 100644 --- a/docs/my-website/release_notes/v1.77.5-stable/index.md +++ b/docs/my-website/release_notes/v1.77.5-stable/index.md @@ -59,6 +59,8 @@ pip install litellm==1.77.5 ### Performance Improvements - 54% RPS Improvement + + Throughput increased by 54% (1,040 → 1,602 RPS, aggregated) per instance while maintaining a 40 ms median overhead. The improvement comes from fixing major O(n²) inefficiencies in the router, primarily caused by repeated use of in statements inside loops over large arrays. Tests were run with a database-only setup (no cache hits). As a result, p95 latency improved by 30% (2,700 → 1,900 ms), enhancing overall stability and scalability under heavy load. #### Test Setup diff --git a/docs/my-website/release_notes/v1.77.7-stable/index.md b/docs/my-website/release_notes/v1.77.7-stable/index.md index 2bd58b54ec7..2631a7bbfb6 100644 --- a/docs/my-website/release_notes/v1.77.7-stable/index.md +++ b/docs/my-website/release_notes/v1.77.7-stable/index.md @@ -67,7 +67,9 @@ pip install litellm==1.77.7.rc.1 ### 2.9x Lower Median Latency - + + +
This update removes LiteLLM router inefficiencies, reducing complexity from O(M×N) to O(1). Previously, it built a new array and ran repeated checks like data["model"] in llm_router.get_model_ids(). Now, a direct ID-to-deployment map eliminates redundant allocations and scans.