scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

docs for deep infra

Browse source
This commit is contained in:
ishaan-jaff 2023-09-26 13:32:55 -07:00
parent a72880925c
commit fef4146396
2 changed files with 39 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

38
docs/my-website/docs/providers/deepinfra.md Normal file
View file

@ -0,0 +1,38 @@
# DeepInfra
https://deepinfra.com/
## Sample Usage
```python
import litellm
import os
os.environ['DEEPINFRA_API_KEY'] = ""
response = completion(
model="deepinfra/meta-llama/Llama-2-70b-chat-hf",
messages=[{"role": "user", "content": "write code for saying hi from LiteLLM"}]
)
```
## Sample Usage - Streaming
```python
import litellm
import os
os.environ['DEEPINFRA_API_KEY'] = ""
response = completion(
model="deepinfra/meta-llama/Llama-2-70b-chat-hf",
messages=[{"role": "user", "content": "write code for saying hi from LiteLLM"}],
stream=True
)
for chunk in response:
print(chunk)
```
## Chat Models
| Model Name | Function Call | Required OS Variables |
|------------------|--------------------------------------|-------------------------|
| meta-llama/Llama-2-70b-chat-hf | `completion(model="deepinfra/meta-llama/Llama-2-70b-chat-hf", messages)` | `os.environ['DEEPINFRA_API_KEY']` |
| meta-llama/Llama-2-7b-chat-hf | `completion(model="deepinfra/meta-llama/Llama-2-7b-chat-hf", messages)` | `os.environ['DEEPINFRA_API_KEY']` |
| codellama/CodeLlama-34b-Instruct-hf | `completion(model="deepinfra/codellama/CodeLlama-34b-Instruct-hf", messages)` | `os.environ['DEEPINFRA_API_KEY']` |

1
docs/my-website/sidebars.js
View file

@ -61,6 +61,7 @@ const sidebars = {
"providers/palm",
"providers/anthropic",
"providers/vllm",
"providers/deepinfra",
"providers/ai21",
"providers/nlp_cloud",
"providers/replicate",