scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 115

(docs) using mistral azure ai studio

Browse source
This commit is contained in:
ishaan-jaff 2024-02-29 08:14:24 -08:00
parent d1f4c83c31
commit 46134d3fba
1 changed files with 23 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/my-website/docs/providers/azure_ai.md
View file

@ -2,16 +2,35 @@
## Using Mistral models deployed on Azure AI Studio
**Ensure you have the `/v1` in your api_base**
### Sample Usage - setting env vars
Set `MISTRAL_API_KEY` and `MISTRAL_API_BASE` in your env
```shell
MISTRAL_API_KEY = "zE************"
MISTRAL_API_BASE = "https://Mistral-large-nmefg-serverless.eastus2.inference.ai.azure.com"
```
### Sample Usage
```python
from litellm import completion
import os
response = completion(
model="mistral/Mistral-large-dfgfj",
api_base="https://Mistral-large-dfgfj-serverless.eastus2.inference.ai.azure.com/v1",
messages=[
{"role": "user", "content": "hello from litellm"}
],
)
print(response)
```
### Sample Usage - passing `api_base` and `api_key` to `litellm.completion`
```python
from litellm import completion
import os
response = completion(
model="mistral/Mistral-large-dfgfj",
api_base="https://Mistral-large-dfgfj-serverless.eastus2.inference.ai.azure.com",
api_key = "JGbKodRcTp****"
messages=[
{"role": "user", "content": "hello from litellm"}
@ -23,14 +42,12 @@ print(response)
### [LiteLLM Proxy] Using Mistral Models
Set this on your litellm proxy config.yaml
**Ensure you have the `/v1` in your api_base**
```yaml
model_list:
- model_name: mistral
litellm_params:
model: mistral/Mistral-large-dfgfj
api_base: https://Mistral-large-dfgfj-serverless.eastus2.inference.ai.azure.com/v1
api_base: https://Mistral-large-dfgfj-serverless.eastus2.inference.ai.azure.com
api_key: JGbKodRcTp****
```