diff --git a/docs/my-website/docs/completion/stream.md b/docs/my-website/docs/completion/stream.md index 491a97ca54a..088437a76d9 100644 --- a/docs/my-website/docs/completion/stream.md +++ b/docs/my-website/docs/completion/stream.md @@ -3,9 +3,11 @@ import TabItem from '@theme/TabItem'; # Streaming + Async -- [Streaming Responses](#streaming-responses) -- [Async Completion](#async-completion) -- [Async + Streaming Completion](#async-streaming) +| Feature | LiteLLM SDK | LiteLLM Proxy | +|---------|-------------|---------------| +| Streaming | ✅ [start here](#streaming-responses) | ✅ [start here](../proxy/user_keys#streaming) | +| Async | ✅ [start here](#async-completion) | ✅ [start here](../proxy/user_keys#streaming) | +| Async Streaming | ✅ [start here](#async-streaming) | ✅ [start here](../proxy/user_keys#streaming) | ## Streaming Responses LiteLLM supports streaming the model response back by passing `stream=True` as an argument to the completion function diff --git a/docs/my-website/docs/proxy/user_keys.md b/docs/my-website/docs/proxy/user_keys.md index 08a5ff04a0a..e56cc6867df 100644 --- a/docs/my-website/docs/proxy/user_keys.md +++ b/docs/my-website/docs/proxy/user_keys.md @@ -381,6 +381,51 @@ assert user.age == 25 ``` +### **Streaming** + + + + + +```bash +curl http://0.0.0.0:4000/v1/chat/completions \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer $OPTIONAL_YOUR_PROXY_KEY" \ +-d '{ + "model": "gpt-4-turbo", + "messages": [ + { + "role": "user", + "content": "this is a test request, write a short poem" + } + ], + "stream": true +}' +``` + + + +```python +from openai import OpenAI +client = OpenAI( + api_key="sk-1234", # [OPTIONAL] set if you set one on proxy, else set "" + base_url="http://0.0.0.0:4000", +) + +messages = [{"role": "user", "content": "this is a test request, write a short poem"}] +completion = client.chat.completions.create( + model="gpt-4o", + messages=messages, + stream=True +) + +print(completion) + +``` + + + + ### Function Calling Here's some examples of doing function calling with the proxy. diff --git a/docs/my-website/docusaurus.config.js b/docs/my-website/docusaurus.config.js index 7a45eff4c71..090a8d0ebb5 100644 --- a/docs/my-website/docusaurus.config.js +++ b/docs/my-website/docusaurus.config.js @@ -43,12 +43,6 @@ const config = { id: 'release_notes', path: './release_notes', // Folder where your release notes are stored routeBasePath: '/release_notes', // URL path for the release notes - sortPosts: (a, b) => { - // Extract folder names from the file paths - const folderA = a.metadata.permalink.split('/')[2]; // Get folder name from permalink - const folderB = b.metadata.permalink.split('/')[2]; - return folderA.localeCompare(folderB); // Compare folder names - }, include: ['**/*.md', '**/*.mdx'], // Files to include // Other blog options },