Add docs

2026-09-12 23:01:41 +00:00 · 2026-03-13 17:47:38 +05:30 · 2026-03-13 17:47:38 +05:30 · 12e44dfe83
commit 12e44dfe83
parent ca996e18e3
7 changed files with 542 additions and 0 deletions
--- a/docs/my-website/blog/mcp_progress_notifications/index.md
+++ b/docs/my-website/blog/mcp_progress_notifications/index.md
@ -0,0 +1,111 @@
+---
+slug: mcp_progress_notifications
+title: "MCP Progress Notifications: Stateless and Stateful Clients on LiteLLM"
+date: 2026-03-13T10:00:00
+authors:
+  - name: Sameer Kankute
+    title: SWE @ LiteLLM (LLM Translation)
+    url: https://www.linkedin.com/in/sameer-kankute/
+    image_url: https://pbs.twimg.com/profile_images/2001352686994907136/ONgNuSk5_400x400.jpg
+  - name: Krrish Dholakia
+    title: "CEO, LiteLLM"
+    url: https://www.linkedin.com/in/krish-d/
+    image_url: https://pbs.twimg.com/profile_images/1298587542745358340/DZv3Oj-h_400x400.jpg
+  - name: Ishaan Jaff
+    title: "CTO, LiteLLM"
+    url: https://www.linkedin.com/in/reffajnaahsi/
+    image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
+description: "LiteLLM now supports both stateless (curl, Inspector) and stateful (Claude Code, Cursor, VSCode) MCP clients on the same endpoint, with progress notifications for long-running tools."
+tags: [mcp, progress, streamable-http, cursor, vscode]
+hide_table_of_contents: false
+---
+
+LiteLLM Proxy's MCP Gateway now supports **both stateless and stateful Streamable HTTP clients** on the same endpoint. curl and MCP Inspector work without changes, while Claude Code, Cursor, and VSCode get session IDs and **progress notifications** for long-running tools.
+
+## Architecture
+
+![MCP Architecture: Stateless and Stateful routing](../../img/litellm_mcp_hybrid_routing_v2.svg)
+
+## The Problem
+
+Previously, LiteLLM used a single stateless session manager. That meant:
+
+- **curl, Inspector, Notion** - No session ID needed
+- **Progress notifications** - Silently failed (no session to push to)
+- **mcp-session-id** - Never returned to clients
+
+Stateful clients (IDEs) need sessions to receive `notifications/progress` during tool execution. Stateless clients don't. We needed both.
+
+## The Solution: Automatic Routing
+
+LiteLLM now routes requests based on the `mcp-session-id` header and request method:
+
+| Scenario | Route | Result |
+|----------|-------|--------|
+| `initialize` (no session) | Stateful | Client gets `mcp-session-id` |
+| Request with `mcp-session-id` | Stateful | Progress notifications work |
+| Other (no session) | Stateless | curl, Inspector work as before |
+
+No configuration required—routing is automatic.
+
+## Progress Flow
+
+![MCP Process flow](../../img/mcp_progress_sequence_v2.svg)
+
+## Quick Test
+
+**1. Initialize** - Get `mcp-session-id` from response headers.
+
+**2. Open GET channel (Terminal A)** - Connect with `Accept: text/event-stream` and `mcp-session-id`.
+
+**3. Call progress tool (Terminal B)** - POST `tools/call` with `_meta.progressToken`.
+
+Progress events stream in Terminal A while Terminal B waits for the final result.
+
+<details>
+<summary>Full code example</summary>
+
+```bash
+# 1. Initialize, get session ID
+SESSION_ID=$(curl -s -D - http://localhost:4000/mcp/progress_test \
+  -H "Content-Type: application/json" \
+  -H "x-litellm-api-key: Bearer sk-1234" \
+  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' \
+  | grep -i mcp-session-id | cut -d' ' -f2 | tr -d '\r')
+
+# 2. Open GET channel (Terminal A)
+curl -N http://localhost:4000/mcp/progress_test \
+  -H "Accept: text/event-stream" \
+  -H "x-litellm-api-key: Bearer sk-1234" \
+  -H "mcp-session-id: $SESSION_ID"
+
+# 3. Call progress tool (Terminal B)
+curl -N http://localhost:4000/mcp/progress_test \
+  -H "Content-Type: application/json" \
+  -H "x-litellm-api-key: Bearer sk-1234" \
+  -H "mcp-session-id: $SESSION_ID" \
+  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"progress_test-progress_tool","arguments":{"steps":3},"_meta":{"progressToken":"tok-123"}}}'
+```
+
+</details>
+
+<iframe width="740" height="500" src="https://www.loom.com/embed/8d83322e7b3247818f8b30e431d30049?autoplay=1" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+
+## Stale Sessions
+
+If LiteLLM restarts, clients may reconnect with an old `mcp-session-id`. LiteLLM strips the stale header and treats the request as a new connection—no 400 errors. For `initialize`, the client gets a fresh session ID.
+
+## FAQ
+
+**Q: When do I need to send `mcp-session-id`?**  
+A: Only if you want progress notifications. Stateless clients (curl, Inspector) can omit it and work as before.
+
+**Q: Do I need to configure stateless vs stateful routing?**  
+A: No. LiteLLM routes automatically based on the `mcp-session-id` header and whether the request is `initialize`.
+
+**Q: I'm not seeing progress events. What's wrong?**  
+A: Ensure you (1) call `initialize` first and capture the `mcp-session-id` from response headers, (2) open a GET connection with `Accept: text/event-stream` and the same session ID, and (3) include `_meta.progressToken` in your `tools/call` params.
+
+**Q: What happens if I use an old session ID after LiteLLM restarts?**  
+A: LiteLLM treats it as a new connection. Send `initialize` again to get a fresh session ID.
+
--- a/docs/my-website/docs/mcp_streamable_http.md
+++ b/docs/my-website/docs/mcp_streamable_http.md
@ -0,0 +1,100 @@
+# Streamable HTTP: Stateless and Stateful Clients
+
+LiteLLM supports both **stateless** and **stateful** Streamable HTTP clients on the same endpoint. curl and MCP Inspector work without changes, while Claude Code, Cursor, and VSCode get session IDs and **progress notifications** for long-running tools.
+
+## Architecture
+
+![MCP Architecture: Stateless and Stateful routing](/img/litellm_mcp_hybrid_routing_v2.svg)
+
+## The Problem
+
+Previously, LiteLLM used a single stateless session manager. That meant:
+
+- **curl, Inspector, Notion** - No session ID needed
+- **Progress notifications** - Silently failed (no session to push to)
+- **mcp-session-id** - Never returned to clients
+
+Stateful clients (IDEs) need sessions to receive `notifications/progress` during tool execution. Stateless clients don't. We needed both.
+
+## The Solution: Automatic Routing
+
+LiteLLM now routes requests based on the `mcp-session-id` header and request method:
+
+| Scenario | Route | Result |
+|----------|-------|--------|
+| `initialize` (no session) | Stateful | Client gets `mcp-session-id` |
+| Request with `mcp-session-id` | Stateful | Progress notifications work |
+| Other (no session) | Stateless | curl, Inspector work as before |
+
+No configuration required—routing is automatic.
+
+## Progress Flow
+
+![MCP Progress Flow](/img/mcp_progress_sequence_v2.svg)
+
+## Quick Test
+
+**1. Initialize** - Get `mcp-session-id` from response headers.
+
+**2. Open GET channel (Terminal A)** - Connect with `Accept: text/event-stream` and `mcp-session-id`.
+
+**3. Call progress tool (Terminal B)** - POST `tools/call` with `_meta.progressToken`.
+
+Progress events stream in Terminal A while Terminal B waits for the final result.
+
+<details>
+<summary>Full code example</summary>
+
+```bash
+# 1. Initialize, get session ID
+SESSION_ID=$(curl -s -D - http://localhost:4000/mcp/progress_test \
+  -H "Content-Type: application/json" \
+  -H "x-litellm-api-key: Bearer sk-1234" \
+  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' \
+  | grep -i mcp-session-id | cut -d' ' -f2 | tr -d '\r')
+
+# 2. Open GET channel (Terminal A)
+curl -N http://localhost:4000/mcp/progress_test \
+  -H "Accept: text/event-stream" \
+  -H "x-litellm-api-key: Bearer sk-1234" \
+  -H "mcp-session-id: $SESSION_ID"
+
+# 3. Call progress tool (Terminal B)
+curl -N http://localhost:4000/mcp/progress_test \
+  -H "Content-Type: application/json" \
+  -H "x-litellm-api-key: Bearer sk-1234" \
+  -H "mcp-session-id: $SESSION_ID" \
+  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"progress_test-progress_tool","arguments":{"steps":3},"_meta":{"progressToken":"tok-123"}}}'
+```
+
+</details>
+
+<iframe width="840" height="500" src="https://www.loom.com/embed/8d83322e7b3247818f8b30e431d30049?autoplay=1" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+
+## Stale Sessions
+
+If LiteLLM restarts, clients may reconnect with an old `mcp-session-id`. LiteLLM strips the stale header and treats the request as a new connection—no 400 errors. For `initialize`, the client gets a fresh session ID.
+
+## FAQ
+
+**Q: When do I need to send `mcp-session-id`?**
+
+Only if you want progress notifications. Stateless clients (curl, Inspector) can omit it and work as before.
+
+**Q: Do I need to configure stateless vs stateful routing?**
+
+No. LiteLLM routes automatically based on the `mcp-session-id` header and whether the request is `initialize`.
+
+**Q: I'm not seeing progress events. What's wrong?**
+
+Ensure you (1) call `initialize` first and capture the `mcp-session-id` from response headers, (2) open a GET connection with `Accept: text/event-stream` and the same session ID, and (3) include `_meta.progressToken` in your `tools/call` params.
+
+**Q: What happens if I use an old session ID after LiteLLM restarts?**
+
+LiteLLM treats it as a new connection. Send `initialize` again to get a fresh session ID.
+
+## Learn More
+
+- [MCP Overview](/docs/mcp) — Full MCP docs
+- [MCP Progress Notifications blog post](/blog/mcp_progress_notifications) — Deeper dive with architecture
+- [MCP Troubleshooting](/docs/mcp_troubleshoot) — Debug connectivity issues
--- a/docs/my-website/img/litellm_mcp_hybrid_routing_v2.svg
+++ b/docs/my-website/img/litellm_mcp_hybrid_routing_v2.svg
@ -0,0 +1,79 @@
+<svg width="100%" viewBox="0 0 680 420" xmlns="http://www.w3.org/2000/svg">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#888" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <marker id="arr-dash" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#aaa" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <style>
+      text { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+    </style>
+  </defs>
+
+  <!-- ── Clients ── -->
+  <rect x="30" y="80" width="140" height="40" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="100" y="105" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">curl / Inspector</text>
+
+  <rect x="30" y="290" width="140" height="40" rx="8" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="100" y="315" text-anchor="middle" font-size="13" font-weight="500" fill="#26215C">VSCode / Cursor</text>
+
+  <!-- ── LiteLLM proxy dashed border ── -->
+  <rect x="235" y="55" width="210" height="310" rx="12" fill="none" stroke="#ccc" stroke-width="1" stroke-dasharray="5 4"/>
+  <text x="340" y="42" text-anchor="middle" font-size="11" fill="#888">LiteLLM proxy</text>
+
+  <!-- ── Router ── -->
+  <rect x="260" y="170" width="160" height="40" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="340" y="195" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">Request router</text>
+
+  <!-- ── Stateless manager ── -->
+  <rect x="252" y="255" width="176" height="52" rx="8" fill="#E1F5EE" stroke="#5DCAA5" stroke-width="1"/>
+  <text x="340" y="276" text-anchor="middle" font-size="13" font-weight="500" fill="#085041">Stateless manager</text>
+  <text x="340" y="295" text-anchor="middle" font-size="11" fill="#0F6E56">no session ID</text>
+
+  <!-- ── Stateful manager ── -->
+  <rect x="252" y="330" width="176" height="52" rx="8" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="340" y="351" text-anchor="middle" font-size="13" font-weight="500" fill="#26215C">Stateful manager</text>
+  <text x="340" y="370" text-anchor="middle" font-size="11" fill="#534AB7">issues mcp-session-id</text>
+
+  <!-- ── Backend ── -->
+  <rect x="510" y="275" width="140" height="40" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="580" y="300" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">Backend MCP</text>
+
+  <!-- ── Arrows ── -->
+
+  <!-- curl → router -->
+  <path d="M170 100 L215 100 L215 190 L258 190" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- VSCode → router -->
+  <path d="M170 310 L215 310 L215 200 L258 200" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- router → stateless -->
+  <line x1="340" y1="210" x2="340" y2="253" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="355" y="236" font-size="11" fill="#555">no session ID</text>
+
+  <!-- router → stateful (right side detour) -->
+  <path d="M420 190 L455 190 L455 356 L430 356" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="458" y="255" font-size="11" fill="#555">initialize or</text>
+  <text x="458" y="270" font-size="11" fill="#555">has session ID</text>
+
+  <!-- stateless → backend -->
+  <line x1="428" y1="281" x2="508" y2="291" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- stateful → backend -->
+  <path d="M428 356 L468 356 L468 315 L508 305" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- backend → stateful: progress (dashed) -->
+  <path d="M510 310 L468 310 L468 330 L430 348" fill="none" stroke="#aaa" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-dash)"/>
+  <text x="472" y="325" font-size="11" fill="#888">progress</text>
+
+  <!-- stateful → VSCode: session ID (dashed) -->
+  <path d="M252 356 L205 356 L205 330 L170 330" fill="none" stroke="#aaa" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-dash)"/>
+  <text x="100" y="375" text-anchor="middle" font-size="11" fill="#888">mcp-session-id</text>
+
+  <!-- Legend -->
+  <line x1="30" y1="408" x2="55" y2="408" stroke="#888" stroke-width="1.5"/>
+  <text x="60" y="412" font-size="11" fill="#555">request</text>
+  <line x1="130" y1="408" x2="155" y2="408" stroke="#aaa" stroke-width="1" stroke-dasharray="4 3"/>
+  <text x="160" y="412" font-size="11" fill="#555">response / notification</text>
+</svg>
--- a/docs/my-website/img/mcp_progress_sequence_v2.svg
+++ b/docs/my-website/img/mcp_progress_sequence_v2.svg
@ -0,0 +1,86 @@
+<svg width="100%" viewBox="0 0 680 520" xmlns="http://www.w3.org/2000/svg">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#555" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <marker id="arr-d" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#7B68EE" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <style>text { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }</style>
+  </defs>
+
+  <!-- Column headers -->
+  <rect x="20"  y="20" width="120" height="36" rx="8" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="80"  y="43" text-anchor="middle" font-size="13" font-weight="500" fill="#26215C">Client</text>
+
+  <rect x="270" y="20" width="140" height="36" rx="8" fill="#E1F5EE" stroke="#5DCAA5" stroke-width="1"/>
+  <text x="340" y="43" text-anchor="middle" font-size="13" font-weight="500" fill="#085041">LiteLLM proxy</text>
+
+  <rect x="530" y="20" width="130" height="36" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="595" y="43" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">Backend MCP</text>
+
+  <!-- Lifelines -->
+  <line x1="80"  y1="56" x2="80"  y2="510" stroke="#ddd" stroke-width="1" stroke-dasharray="4 3"/>
+  <line x1="340" y1="56" x2="340" y2="510" stroke="#ddd" stroke-width="1" stroke-dasharray="4 3"/>
+  <line x1="595" y1="56" x2="595" y2="510" stroke="#ddd" stroke-width="1" stroke-dasharray="4 3"/>
+
+  <!-- 1: initialize -->
+  <line x1="80" y1="95" x2="335" y2="95" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="88" text-anchor="middle" font-size="11" fill="#333">POST initialize</text>
+  <line x1="340" y1="108" x2="85" y2="108" stroke="#aaa" stroke-width="1" stroke-dasharray="3 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="122" text-anchor="middle" font-size="11" fill="#7B68EE">← mcp-session-id: abc123</text>
+
+  <!-- 2: GET SSE channel — shown as thick persistent bar -->
+  <line x1="80" y1="150" x2="335" y2="150" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="143" text-anchor="middle" font-size="11" fill="#333">GET /mcp  (mcp-session-id: abc123)</text>
+
+  <!-- SSE channel open band -->
+  <rect x="68" y="158" width="24" height="230" rx="4" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="80" y="278" text-anchor="middle" font-size="9" fill="#534AB7" transform="rotate(-90,80,278)">SSE channel open</text>
+  <rect x="328" y="158" width="24" height="230" rx="4" fill="#E1F5EE" stroke="#5DCAA5" stroke-width="1"/>
+
+  <!-- 3: tools/call POST -->
+  <line x1="92" y1="200" x2="328" y2="200" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="193" text-anchor="middle" font-size="11" fill="#333">POST tools/call + progressToken</text>
+  <line x1="352" y1="213" x2="528" y2="213" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="440" y="206" text-anchor="middle" font-size="11" fill="#333">forward tools/call</text>
+
+  <!-- backend running -->
+  <text x="595" y="245" text-anchor="middle" font-size="10" fill="#aaa">running...</text>
+
+  <!-- progress 1/3 -->
+  <line x1="528" y1="258" x2="354" y2="258" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="440" y="251" text-anchor="middle" font-size="11" fill="#7B68EE">progress 1/3</text>
+  <line x1="328" y1="268" x2="92" y2="268" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="261" text-anchor="middle" font-size="11" fill="#7B68EE">notifications/progress {1, 3}</text>
+
+  <!-- progress 2/3 -->
+  <line x1="528" y1="300" x2="354" y2="300" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="440" y="293" text-anchor="middle" font-size="11" fill="#7B68EE">progress 2/3</text>
+  <line x1="328" y1="310" x2="92" y2="310" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="303" text-anchor="middle" font-size="11" fill="#7B68EE">notifications/progress {2, 3}</text>
+
+  <!-- progress 3/3 -->
+  <line x1="528" y1="342" x2="354" y2="342" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="440" y="335" text-anchor="middle" font-size="11" fill="#7B68EE">progress 3/3</text>
+  <line x1="328" y1="352" x2="92" y2="352" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="345" text-anchor="middle" font-size="11" fill="#7B68EE">notifications/progress {3, 3}</text>
+
+  <!-- SSE channel close -->
+  <line x1="68" y1="388" x2="92" y2="388" stroke="#AFA9EC" stroke-width="1"/>
+  <line x1="328" y1="388" x2="352" y2="388" stroke="#5DCAA5" stroke-width="1"/>
+
+  <!-- final result -->
+  <line x1="528" y1="410" x2="354" y2="410" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="440" y="403" text-anchor="middle" font-size="11" fill="#333">tool result</text>
+  <line x1="328" y1="423" x2="92" y2="423" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="416" text-anchor="middle" font-size="11" fill="#333">POST response: final result</text>
+
+  <!-- Legend -->
+  <line x1="30" y1="500" x2="52" y2="500" stroke="#555" stroke-width="1.5"/>
+  <text x="57" y="504" font-size="11" fill="#555">request / result</text>
+  <line x1="175" y1="500" x2="197" y2="500" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3"/>
+  <text x="202" y="504" font-size="11" fill="#7B68EE">progress notification</text>
+  <rect x="380" y="492" width="12" height="14" rx="2" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="397" y="504" font-size="11" fill="#534AB7">SSE channel</text>
+</svg>
--- a/docs/my-website/sidebars.js
+++ b/docs/my-website/sidebars.js
@ -627,6 +627,7 @@ const sidebars = {
          label: "/mcp - Model Context Protocol",
          items: [
            "mcp",
+            "mcp_streamable_http",
            "mcp_usage",
            "mcp_openapi",
            "mcp_oauth",
--- a/docs/my-website/static/img/litellm_mcp_hybrid_routing_v2.svg
+++ b/docs/my-website/static/img/litellm_mcp_hybrid_routing_v2.svg
@ -0,0 +1,79 @@
+<svg width="100%" viewBox="0 0 680 420" xmlns="http://www.w3.org/2000/svg">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#888" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <marker id="arr-dash" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#aaa" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <style>
+      text { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+    </style>
+  </defs>
+
+  <!-- ── Clients ── -->
+  <rect x="30" y="80" width="140" height="40" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="100" y="105" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">curl / Inspector</text>
+
+  <rect x="30" y="290" width="140" height="40" rx="8" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="100" y="315" text-anchor="middle" font-size="13" font-weight="500" fill="#26215C">VSCode / Cursor</text>
+
+  <!-- ── LiteLLM proxy dashed border ── -->
+  <rect x="235" y="55" width="210" height="310" rx="12" fill="none" stroke="#ccc" stroke-width="1" stroke-dasharray="5 4"/>
+  <text x="340" y="42" text-anchor="middle" font-size="11" fill="#888">LiteLLM proxy</text>
+
+  <!-- ── Router ── -->
+  <rect x="260" y="170" width="160" height="40" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="340" y="195" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">Request router</text>
+
+  <!-- ── Stateless manager ── -->
+  <rect x="252" y="255" width="176" height="52" rx="8" fill="#E1F5EE" stroke="#5DCAA5" stroke-width="1"/>
+  <text x="340" y="276" text-anchor="middle" font-size="13" font-weight="500" fill="#085041">Stateless manager</text>
+  <text x="340" y="295" text-anchor="middle" font-size="11" fill="#0F6E56">no session ID</text>
+
+  <!-- ── Stateful manager ── -->
+  <rect x="252" y="330" width="176" height="52" rx="8" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="340" y="351" text-anchor="middle" font-size="13" font-weight="500" fill="#26215C">Stateful manager</text>
+  <text x="340" y="370" text-anchor="middle" font-size="11" fill="#534AB7">issues mcp-session-id</text>
+
+  <!-- ── Backend ── -->
+  <rect x="510" y="275" width="140" height="40" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="580" y="300" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">Backend MCP</text>
+
+  <!-- ── Arrows ── -->
+
+  <!-- curl → router -->
+  <path d="M170 100 L215 100 L215 190 L258 190" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- VSCode → router -->
+  <path d="M170 310 L215 310 L215 200 L258 200" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- router → stateless -->
+  <line x1="340" y1="210" x2="340" y2="253" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="355" y="236" font-size="11" fill="#555">no session ID</text>
+
+  <!-- router → stateful (right side detour) -->
+  <path d="M420 190 L455 190 L455 356 L430 356" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="458" y="255" font-size="11" fill="#555">initialize or</text>
+  <text x="458" y="270" font-size="11" fill="#555">has session ID</text>
+
+  <!-- stateless → backend -->
+  <line x1="428" y1="281" x2="508" y2="291" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- stateful → backend -->
+  <path d="M428 356 L468 356 L468 315 L508 305" fill="none" stroke="#888" stroke-width="1.5" marker-end="url(#arr)"/>
+
+  <!-- backend → stateful: progress (dashed) -->
+  <path d="M510 310 L468 310 L468 330 L430 348" fill="none" stroke="#aaa" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-dash)"/>
+  <text x="472" y="325" font-size="11" fill="#888">progress</text>
+
+  <!-- stateful → VSCode: session ID (dashed) -->
+  <path d="M252 356 L205 356 L205 330 L170 330" fill="none" stroke="#aaa" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-dash)"/>
+  <text x="100" y="375" text-anchor="middle" font-size="11" fill="#888">mcp-session-id</text>
+
+  <!-- Legend -->
+  <line x1="30" y1="408" x2="55" y2="408" stroke="#888" stroke-width="1.5"/>
+  <text x="60" y="412" font-size="11" fill="#555">request</text>
+  <line x1="130" y1="408" x2="155" y2="408" stroke="#aaa" stroke-width="1" stroke-dasharray="4 3"/>
+  <text x="160" y="412" font-size="11" fill="#555">response / notification</text>
+</svg>
--- a/docs/my-website/static/img/mcp_progress_sequence_v2.svg
+++ b/docs/my-website/static/img/mcp_progress_sequence_v2.svg
@ -0,0 +1,86 @@
+<svg width="100%" viewBox="0 0 680 520" xmlns="http://www.w3.org/2000/svg">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#555" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <marker id="arr-d" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
+      <path d="M2 1L8 5L2 9" fill="none" stroke="#7B68EE" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
+    </marker>
+    <style>text { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }</style>
+  </defs>
+
+  <!-- Column headers -->
+  <rect x="20"  y="20" width="120" height="36" rx="8" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="80"  y="43" text-anchor="middle" font-size="13" font-weight="500" fill="#26215C">Client</text>
+
+  <rect x="270" y="20" width="140" height="36" rx="8" fill="#E1F5EE" stroke="#5DCAA5" stroke-width="1"/>
+  <text x="340" y="43" text-anchor="middle" font-size="13" font-weight="500" fill="#085041">LiteLLM proxy</text>
+
+  <rect x="530" y="20" width="130" height="36" rx="8" fill="#F1EFE8" stroke="#B4B2A9" stroke-width="1"/>
+  <text x="595" y="43" text-anchor="middle" font-size="13" font-weight="500" fill="#2C2C2A">Backend MCP</text>
+
+  <!-- Lifelines -->
+  <line x1="80"  y1="56" x2="80"  y2="510" stroke="#ddd" stroke-width="1" stroke-dasharray="4 3"/>
+  <line x1="340" y1="56" x2="340" y2="510" stroke="#ddd" stroke-width="1" stroke-dasharray="4 3"/>
+  <line x1="595" y1="56" x2="595" y2="510" stroke="#ddd" stroke-width="1" stroke-dasharray="4 3"/>
+
+  <!-- 1: initialize -->
+  <line x1="80" y1="95" x2="335" y2="95" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="88" text-anchor="middle" font-size="11" fill="#333">POST initialize</text>
+  <line x1="340" y1="108" x2="85" y2="108" stroke="#aaa" stroke-width="1" stroke-dasharray="3 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="122" text-anchor="middle" font-size="11" fill="#7B68EE">← mcp-session-id: abc123</text>
+
+  <!-- 2: GET SSE channel — shown as thick persistent bar -->
+  <line x1="80" y1="150" x2="335" y2="150" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="143" text-anchor="middle" font-size="11" fill="#333">GET /mcp  (mcp-session-id: abc123)</text>
+
+  <!-- SSE channel open band -->
+  <rect x="68" y="158" width="24" height="230" rx="4" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="80" y="278" text-anchor="middle" font-size="9" fill="#534AB7" transform="rotate(-90,80,278)">SSE channel open</text>
+  <rect x="328" y="158" width="24" height="230" rx="4" fill="#E1F5EE" stroke="#5DCAA5" stroke-width="1"/>
+
+  <!-- 3: tools/call POST -->
+  <line x1="92" y1="200" x2="328" y2="200" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="193" text-anchor="middle" font-size="11" fill="#333">POST tools/call + progressToken</text>
+  <line x1="352" y1="213" x2="528" y2="213" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="440" y="206" text-anchor="middle" font-size="11" fill="#333">forward tools/call</text>
+
+  <!-- backend running -->
+  <text x="595" y="245" text-anchor="middle" font-size="10" fill="#aaa">running...</text>
+
+  <!-- progress 1/3 -->
+  <line x1="528" y1="258" x2="354" y2="258" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="440" y="251" text-anchor="middle" font-size="11" fill="#7B68EE">progress 1/3</text>
+  <line x1="328" y1="268" x2="92" y2="268" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="261" text-anchor="middle" font-size="11" fill="#7B68EE">notifications/progress {1, 3}</text>
+
+  <!-- progress 2/3 -->
+  <line x1="528" y1="300" x2="354" y2="300" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="440" y="293" text-anchor="middle" font-size="11" fill="#7B68EE">progress 2/3</text>
+  <line x1="328" y1="310" x2="92" y2="310" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="303" text-anchor="middle" font-size="11" fill="#7B68EE">notifications/progress {2, 3}</text>
+
+  <!-- progress 3/3 -->
+  <line x1="528" y1="342" x2="354" y2="342" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="440" y="335" text-anchor="middle" font-size="11" fill="#7B68EE">progress 3/3</text>
+  <line x1="328" y1="352" x2="92" y2="352" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3" marker-end="url(#arr-d)"/>
+  <text x="205" y="345" text-anchor="middle" font-size="11" fill="#7B68EE">notifications/progress {3, 3}</text>
+
+  <!-- SSE channel close -->
+  <line x1="68" y1="388" x2="92" y2="388" stroke="#AFA9EC" stroke-width="1"/>
+  <line x1="328" y1="388" x2="352" y2="388" stroke="#5DCAA5" stroke-width="1"/>
+
+  <!-- final result -->
+  <line x1="528" y1="410" x2="354" y2="410" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="440" y="403" text-anchor="middle" font-size="11" fill="#333">tool result</text>
+  <line x1="328" y1="423" x2="92" y2="423" stroke="#555" stroke-width="1.5" marker-end="url(#arr)"/>
+  <text x="205" y="416" text-anchor="middle" font-size="11" fill="#333">POST response: final result</text>
+
+  <!-- Legend -->
+  <line x1="30" y1="500" x2="52" y2="500" stroke="#555" stroke-width="1.5"/>
+  <text x="57" y="504" font-size="11" fill="#555">request / result</text>
+  <line x1="175" y1="500" x2="197" y2="500" stroke="#7B68EE" stroke-width="1" stroke-dasharray="4 3"/>
+  <text x="202" y="504" font-size="11" fill="#7B68EE">progress notification</text>
+  <rect x="380" y="492" width="12" height="14" rx="2" fill="#EEEDFE" stroke="#AFA9EC" stroke-width="1"/>
+  <text x="397" y="504" font-size="11" fill="#534AB7">SSE channel</text>
+</svg>