+
+
+ 
+
+
+ 
+
+ 
+
+ 
+
++
Call all LLM APIs using the OpenAI format [Anthropic, Huggingface, Cohere, Azure OpenAI etc.]
+ -[](https://discord.gg/wuPM9dRgDw) +
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-Demo - https://litellm.ai/playground
-Docs - https://docs.litellm.ai/docs/
-**Free** Dashboard - https://docs.litellm.ai/docs/debugging/hosted_debugging
-## quick start
```
pip install litellm
```
@@ -28,6 +58,7 @@ from litellm import completion
## set ENV variables
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"
+os.environ["ANTHROPIC_API_KEY"] = "anthropic key"
messages = [{ "content": "Hello, how are you?","role": "user"}]
@@ -35,16 +66,18 @@ messages = [{ "content": "Hello, how are you?","role": "user"}]
response = completion(model="gpt-3.5-turbo", messages=messages)
# cohere call
-response = completion(model="command-nightly", messages)
+response = completion(model="command-nightly", messages=messages)
+
+# anthropic
+response = completion(model="claude-2", messages=messages)
```
-Code Sample: [Getting Started Notebook](https://colab.research.google.com/drive/1gR3pY-JzDZahzpVdbGBtrNGDBmzUNJaJ?usp=sharing)
Stable version
```
pip install litellm==0.1.424
```
-## Streaming Queries
+## Streaming
liteLLM supports streaming the model response back, pass `stream=True` to get a streaming iterator in response.
Streaming is supported for OpenAI, Azure, Anthropic, Huggingface models
```python
@@ -58,11 +91,27 @@ for chunk in result:
print(chunk['choices'][0]['delta'])
```
-# support / talk with founders
-- [Our calendar 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)
+# Support / talk with founders
+- [Schedule Demo 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)
- [Community Discord 💭](https://discord.gg/wuPM9dRgDw)
- Our numbers 📞 +1 (770) 8783-106 / +1 (412) 618-6238
- Our emails ✉️ ishaan@berri.ai / krrish@berri.ai
-# why did we build this
+# Why did we build this
- **Need for simplicity**: Our code started to get extremely complicated managing & translating calls between Azure, OpenAI, Cohere
+
+# Contributors
+
+
+
+
+
+
+
+
+
+
+
+ | Model Name | \n", + "claude-instant-1 | \n", + "gpt-3.5-turbo-0613 | \n", + "gpt-3.5-turbo-16k-0613 | \n", + "gpt-4-0613 | \n", + "replicate/llama-2-70b-chat:58d078176e02c219e11eb4da5a02a7830a283b14cf8f94537af893ccff5ee781 | \n", + "
|---|---|---|---|---|---|
| Prompt | \n", + "\n", + " | \n", + " | \n", + " | \n", + " | \n", + " |
| \\nIs paul graham a writer? | \n", + "Yes, Paul Graham is considered a writer in ad... | \n", + "Yes, Paul Graham is a writer. He has written s... | \n", + "Yes, Paul Graham is a writer. He has authored ... | \n", + "Yes, Paul Graham is a writer. He is an essayis... | \n", + "Yes, Paul Graham is an author. According to t... | \n", + "
| \\nWhat has Paul Graham done? | \n", + "Paul Graham has made significant contribution... | \n", + "Paul Graham has achieved several notable accom... | \n", + "Paul Graham has made significant contributions... | \n", + "Paul Graham is known for his work on the progr... | \n", + "Paul Graham has had a diverse career in compu... | \n", + "
| \\nWhat is Paul Graham known for? | \n", + "Paul Graham is known for several things:\\n\\n-... | \n", + "Paul Graham is known for his work on the progr... | \n", + "Paul Graham is known for his work on the progr... | \n", + "Paul Graham is known for his work on the progr... | \n", + "Paul Graham is known for many things, includi... | \n", + "
| \\nWhere does Paul Graham live? | \n", + "Based on the information provided:\\n\\n- Paul ... | \n", + "According to the given information, Paul Graha... | \n", + "Paul Graham currently lives in England, where ... | \n", + "The text does not provide a current place of r... | \n", + "Based on the information provided, Paul Graha... | \n", + "
| \\nWho is Paul Graham? | \n", + "Paul Graham is an influential computer scient... | \n", + "Paul Graham is an English computer scientist, ... | \n", + "Paul Graham is an English computer scientist, ... | \n", + "Paul Graham is an English computer scientist, ... | \n", + "Paul Graham is an English computer scientist,... | \n", + "
+
Call all LLM APIs using the OpenAI format [Anthropic, Huggingface, Cohere, Azure OpenAI etc.]
+ + +
+
+
+
+
+
+
+
+
+
+
+
+
+
+### 1. Make a normal `completion()` call
```
- import litellm
- from litellm import embedding, completion
-
- ## Set your email
- litellm.email = "test_email@test.com"
-
- user_message = "Hello, how are you?"
- messages = [{ "content": user_message,"role": "user"}]
-
-
- # openai call
- response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}])
-
- # bad request call
- response = completion(model="chatgpt-test", messages=[{"role": "user", "content": "Hi 👋 - i'm a bad request"}])
+pip install litellm
```
+
Demo - https://litellm.ai/playground \
Read the docs - https://docs.litellm.ai/docs/
## quick start
+
```
pip install litellm
```
-```python
-from litellm import completion
+
+ {console.log("token: ", token)}
+ {token ? codeWithToken : codeWithoutToken}
+
+ )
+}
+
+const QueryParamReader = () => {
+ const [token, setToken] = useState(null);
+
+ useEffect(() => {
+ const urlParams = new URLSearchParams(window.location.search);
+ console.log("urlParams: ", urlParams)
+ const token = urlParams.get('token');
+ setToken(token);
+ }, []);
+
+ return (
+