diff --git a/docs/my-website/docs/providers/openai.md b/docs/my-website/docs/providers/openai.md index e3f6c267d15..d40ab0676f5 100644 --- a/docs/my-website/docs/providers/openai.md +++ b/docs/my-website/docs/providers/openai.md @@ -2,7 +2,7 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # OpenAI -LiteLLM supports OpenAI Chat + Text completion and embedding calls. +LiteLLM supports OpenAI Chat + Embedding calls. ### Required API Keys @@ -217,19 +217,6 @@ response = completion( ``` -## OpenAI Text Completion Models / Instruct Models - -| Model Name | Function Call | -|---------------------|----------------------------------------------------| -| gpt-3.5-turbo-instruct | `response = completion(model="gpt-3.5-turbo-instruct", messages=messages)` | -| gpt-3.5-turbo-instruct-0914 | `response = completion(model="gpt-3.5-turbo-instruct-0914", messages=messages)` | -| text-davinci-003 | `response = completion(model="text-davinci-003", messages=messages)` | -| ada-001 | `response = completion(model="ada-001", messages=messages)` | -| curie-001 | `response = completion(model="curie-001", messages=messages)` | -| babbage-001 | `response = completion(model="babbage-001", messages=messages)` | -| babbage-002 | `response = completion(model="babbage-002", messages=messages)` | -| davinci-002 | `response = completion(model="davinci-002", messages=messages)` | - ## Advanced ### Parallel Function calling diff --git a/docs/my-website/docs/providers/openai_compatible.md b/docs/my-website/docs/providers/openai_compatible.md index 09dcd7e4c98..ff0e8570998 100644 --- a/docs/my-website/docs/providers/openai_compatible.md +++ b/docs/my-website/docs/providers/openai_compatible.md @@ -5,7 +5,9 @@ import TabItem from '@theme/TabItem'; To call models hosted behind an openai proxy, make 2 changes: -1. Put `openai/` in front of your model name, so litellm knows you're trying to call an openai-compatible endpoint. +1. For `/chat/completions`: Put `openai/` in front of your model name, so litellm knows you're trying to call an openai `/chat/completions` endpoint. + +2. For `/completions`: Put `text-completion-openai/` in front of your model name, so litellm knows you're trying to call an openai `/completions` endpoint. 2. **Do NOT** add anything additional to the base url e.g. `/v1/embedding`. LiteLLM uses the openai-client to make these calls, and that automatically adds the relevant endpoints. diff --git a/docs/my-website/docs/providers/text_completion_openai.md b/docs/my-website/docs/providers/text_completion_openai.md new file mode 100644 index 00000000000..842b56aec95 --- /dev/null +++ b/docs/my-website/docs/providers/text_completion_openai.md @@ -0,0 +1,163 @@ +# OpenAI (Text Completion) + +LiteLLM supports OpenAI text completion models + +### Required API Keys + +```python +import os +os.environ["OPENAI_API_KEY"] = "your-api-key" +``` + +### Usage +```python +import os +from litellm import completion + +os.environ["OPENAI_API_KEY"] = "your-api-key" + +# openai call +response = completion( + model = "gpt-3.5-turbo-instruct", + messages=[{ "content": "Hello, how are you?","role": "user"}] +) +``` + +### Usage - LiteLLM Proxy Server + +Here's how to call OpenAI models with the LiteLLM Proxy Server + +### 1. Save key in your environment + +```bash +export OPENAI_API_KEY="" +``` + +### 2. Start the proxy + + + + +```yaml +model_list: + - model_name: gpt-3.5-turbo + litellm_params: + model: openai/gpt-3.5-turbo # The `openai/` prefix will call openai.chat.completions.create + api_key: os.environ/OPENAI_API_KEY + - model_name: gpt-3.5-turbo-instruct + litellm_params: + model: text-completion-openai/gpt-3.5-turbo-instruct # The `text-completion-openai/` prefix will call openai.completions.create + api_key: os.environ/OPENAI_API_KEY +``` + + + +Use this to add all openai models with one API Key. **WARNING: This will not do any load balancing** +This means requests to `gpt-4`, `gpt-3.5-turbo` , `gpt-4-turbo-preview` will all go through this route + +```yaml +model_list: + - model_name: "*" # all requests where model not in your config go to this deployment + litellm_params: + model: openai/* # set `openai/` to use the openai route + api_key: os.environ/OPENAI_API_KEY +``` + + + +```bash +$ litellm --model gpt-3.5-turbo-instruct + +# Server running on http://0.0.0.0:4000 +``` + + + + +### 3. Test it + + + + + +```shell +curl --location 'http://0.0.0.0:4000/chat/completions' \ +--header 'Content-Type: application/json' \ +--data ' { + "model": "gpt-3.5-turbo-instruct", + "messages": [ + { + "role": "user", + "content": "what llm are you" + } + ] + } +' +``` + + + +```python +import openai +client = openai.OpenAI( + api_key="anything", + base_url="http://0.0.0.0:4000" +) + +# request sent to model set on litellm proxy, `litellm --model` +response = client.chat.completions.create(model="gpt-3.5-turbo-instruct", messages = [ + { + "role": "user", + "content": "this is a test request, write a short poem" + } +]) + +print(response) + +``` + + + +```python +from langchain.chat_models import ChatOpenAI +from langchain.prompts.chat import ( + ChatPromptTemplate, + HumanMessagePromptTemplate, + SystemMessagePromptTemplate, +) +from langchain.schema import HumanMessage, SystemMessage + +chat = ChatOpenAI( + openai_api_base="http://0.0.0.0:4000", # set openai_api_base to the LiteLLM Proxy + model = "gpt-3.5-turbo-instruct", + temperature=0.1 +) + +messages = [ + SystemMessage( + content="You are a helpful assistant that im using to make a test request to." + ), + HumanMessage( + content="test from litellm. tell me why it's amazing in 1 sentence" + ), +] +response = chat(messages) + +print(response) +``` + + + + +## OpenAI Text Completion Models / Instruct Models + +| Model Name | Function Call | +|---------------------|----------------------------------------------------| +| gpt-3.5-turbo-instruct | `response = completion(model="gpt-3.5-turbo-instruct", messages=messages)` | +| gpt-3.5-turbo-instruct-0914 | `response = completion(model="gpt-3.5-turbo-instruct-0914", messages=messages)` | +| text-davinci-003 | `response = completion(model="text-davinci-003", messages=messages)` | +| ada-001 | `response = completion(model="ada-001", messages=messages)` | +| curie-001 | `response = completion(model="curie-001", messages=messages)` | +| babbage-001 | `response = completion(model="babbage-001", messages=messages)` | +| babbage-002 | `response = completion(model="babbage-002", messages=messages)` | +| davinci-002 | `response = completion(model="davinci-002", messages=messages)` | diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js index 7b8a09196ee..bf1ef6f1498 100644 --- a/docs/my-website/sidebars.js +++ b/docs/my-website/sidebars.js @@ -115,6 +115,7 @@ const sidebars = { }, items: [ "providers/openai", + "providers/text_completion_openai", "providers/openai_compatible", "providers/azure", "providers/azure_ai",