mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-06 02:48:13 +00:00
[Feat] Add Mistral AI reasoning capabilities docs
This commit is contained in:
Cole McIntosh
parent
82334ce522
commit
a30ed8ce0b
2 changed files with 141 additions and 15 deletions
|
|
@ -144,20 +144,22 @@ All models listed here https://docs.mistral.ai/platform/endpoints are supported.
|
|||
:::
|
||||
|
||||
|
||||
| Model Name | Function Call |
|
||||
|----------------|--------------------------------------------------------------|
|
||||
| Mistral Small | `completion(model="mistral/mistral-small-latest", messages)` |
|
||||
| Mistral Medium | `completion(model="mistral/mistral-medium-latest", messages)`|
|
||||
| Mistral Large 2 | `completion(model="mistral/mistral-large-2407", messages)` |
|
||||
| Mistral Large Latest | `completion(model="mistral/mistral-large-latest", messages)` |
|
||||
| Mistral 7B | `completion(model="mistral/open-mistral-7b", messages)` |
|
||||
| Mixtral 8x7B | `completion(model="mistral/open-mixtral-8x7b", messages)` |
|
||||
| Mixtral 8x22B | `completion(model="mistral/open-mixtral-8x22b", messages)` |
|
||||
| Codestral | `completion(model="mistral/codestral-latest", messages)` |
|
||||
| Mistral NeMo | `completion(model="mistral/open-mistral-nemo", messages)` |
|
||||
| Mistral NeMo 2407 | `completion(model="mistral/open-mistral-nemo-2407", messages)` |
|
||||
| Codestral Mamba | `completion(model="mistral/open-codestral-mamba", messages)` |
|
||||
| Codestral Mamba | `completion(model="mistral/codestral-mamba-latest"", messages)` |
|
||||
| Model Name | Function Call | Reasoning Support |
|
||||
|----------------|--------------------------------------------------------------|-------------------|
|
||||
| Mistral Small | `completion(model="mistral/mistral-small-latest", messages)` | ❌ |
|
||||
| Mistral Medium | `completion(model="mistral/mistral-medium-latest", messages)`| ❌ |
|
||||
| Mistral Large 2 | `completion(model="mistral/mistral-large-2407", messages)` | ❌ |
|
||||
| Mistral Large Latest | `completion(model="mistral/mistral-large-latest", messages)` | ❌ |
|
||||
| **Magistral Small** | `completion(model="mistral/magistral-small-2506", messages)` | ✅ |
|
||||
| **Magistral Medium** | `completion(model="mistral/magistral-medium-2506", messages)`| ✅ |
|
||||
| Mistral 7B | `completion(model="mistral/open-mistral-7b", messages)` | ❌ |
|
||||
| Mixtral 8x7B | `completion(model="mistral/open-mixtral-8x7b", messages)` | ❌ |
|
||||
| Mixtral 8x22B | `completion(model="mistral/open-mixtral-8x22b", messages)` | ❌ |
|
||||
| Codestral | `completion(model="mistral/codestral-latest", messages)` | ❌ |
|
||||
| Mistral NeMo | `completion(model="mistral/open-mistral-nemo", messages)` | ❌ |
|
||||
| Mistral NeMo 2407 | `completion(model="mistral/open-mistral-nemo-2407", messages)` | ❌ |
|
||||
| Codestral Mamba | `completion(model="mistral/open-codestral-mamba", messages)` | ❌ |
|
||||
| Codestral Mamba | `completion(model="mistral/codestral-mamba-latest"", messages)` | ❌ |
|
||||
|
||||
## Function Calling
|
||||
|
||||
|
|
@ -203,6 +205,110 @@ assert isinstance(
|
|||
)
|
||||
```
|
||||
|
||||
## Reasoning Capabilities (Magistral Models)
|
||||
|
||||
Mistral's Magistral models support advanced reasoning capabilities that allow the model to think step-by-step before providing answers. LiteLLM provides seamless integration with these reasoning features through OpenAI-compatible parameters.
|
||||
|
||||
### Supported Magistral Models
|
||||
|
||||
| Model Name | Function Call |
|
||||
|----------------|--------------------------------------------------------------|
|
||||
| Magistral Small | `completion(model="mistral/magistral-small-2506", messages)` |
|
||||
| Magistral Medium | `completion(model="mistral/magistral-medium-2506", messages)`|
|
||||
|
||||
### Using Reasoning Effort
|
||||
|
||||
The `reasoning_effort` parameter controls how much effort the model puts into reasoning. When used with magistral models.
|
||||
|
||||
```python
|
||||
from litellm import completion
|
||||
import os
|
||||
|
||||
os.environ['MISTRAL_API_KEY'] = "your-api-key"
|
||||
|
||||
response = completion(
|
||||
model="mistral/magistral-medium-2506",
|
||||
messages=[
|
||||
{"role": "user", "content": "What is 15 multiplied by 7?"}
|
||||
],
|
||||
reasoning_effort="medium" # Options: "low", "medium", "high"
|
||||
)
|
||||
|
||||
print(response)
|
||||
```
|
||||
|
||||
### Example with System Message
|
||||
|
||||
If you already have a system message, LiteLLM will prepend the reasoning instructions:
|
||||
|
||||
```python
|
||||
response = completion(
|
||||
model="mistral/magistral-medium-2506",
|
||||
messages=[
|
||||
{"role": "system", "content": "You are a helpful math tutor."},
|
||||
{"role": "user", "content": "Explain how to solve quadratic equations."}
|
||||
],
|
||||
reasoning_effort="high"
|
||||
)
|
||||
|
||||
# The system message becomes:
|
||||
# "When solving problems, think step-by-step in <think> tags before providing your final answer...
|
||||
#
|
||||
# You are a helpful math tutor."
|
||||
```
|
||||
|
||||
### Usage with LiteLLM Proxy
|
||||
|
||||
You can also use reasoning capabilities through the LiteLLM proxy:
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="Curl" label="Curl Request">
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/chat/completions' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "magistral-medium-2506",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "What is the square root of 144? Show your reasoning."
|
||||
}
|
||||
],
|
||||
"reasoning_effort": "medium"
|
||||
}'
|
||||
```
|
||||
</TabItem>
|
||||
<TabItem value="openai" label="OpenAI v1.0.0+">
|
||||
|
||||
```python
|
||||
import openai
|
||||
client = openai.OpenAI(
|
||||
api_key="anything",
|
||||
base_url="http://0.0.0.0:4000"
|
||||
)
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="magistral-medium-2506",
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Calculate the area of a circle with radius 5. Show your work."
|
||||
}
|
||||
],
|
||||
reasoning_effort="high"
|
||||
)
|
||||
|
||||
print(response)
|
||||
```
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
### Important Notes
|
||||
|
||||
- **Model Compatibility**: Reasoning parameters only work with magistral models
|
||||
- **Backward Compatibility**: Non-magistral models will ignore reasoning parameters and work normally
|
||||
|
||||
## Sample Usage - Embedding
|
||||
```python
|
||||
from litellm import embedding
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ Supported Providers:
|
|||
- Google AI Studio (`google/`)
|
||||
- Vertex AI (`vertex_ai/`)
|
||||
- Perplexity (`perplexity/`)
|
||||
- Mistral AI (Magistral models) (`mistral/`)
|
||||
|
||||
LiteLLM will standardize the `reasoning_content` in the response and `thinking_blocks` in the assistant message.
|
||||
|
||||
|
|
@ -39,7 +40,7 @@ LiteLLM will standardize the `reasoning_content` in the response and `thinking_b
|
|||
## Quick Start
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="sdk" label="SDK">
|
||||
<TabItem value="anthropic" label="Anthropic">
|
||||
|
||||
```python showLineNumbers
|
||||
from litellm import completion
|
||||
|
|
@ -57,6 +58,25 @@ response = completion(
|
|||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="mistral" label="Mistral">
|
||||
|
||||
```python showLineNumbers
|
||||
from litellm import completion
|
||||
import os
|
||||
|
||||
os.environ["MISTRAL_API_KEY"] = ""
|
||||
|
||||
response = completion(
|
||||
model="mistral/magistral-medium-2506",
|
||||
messages=[
|
||||
{"role": "user", "content": "What is 15 multiplied by 7? Show your reasoning."},
|
||||
],
|
||||
reasoning_effort="medium",
|
||||
)
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="proxy" label="PROXY">
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue