diff --git a/docs/my-website/docs/providers/mistral.md b/docs/my-website/docs/providers/mistral.md index 62a91c687ae..b14206a711f 100644 --- a/docs/my-website/docs/providers/mistral.md +++ b/docs/my-website/docs/providers/mistral.md @@ -144,20 +144,22 @@ All models listed here https://docs.mistral.ai/platform/endpoints are supported. ::: -| Model Name | Function Call | -|----------------|--------------------------------------------------------------| -| Mistral Small | `completion(model="mistral/mistral-small-latest", messages)` | -| Mistral Medium | `completion(model="mistral/mistral-medium-latest", messages)`| -| Mistral Large 2 | `completion(model="mistral/mistral-large-2407", messages)` | -| Mistral Large Latest | `completion(model="mistral/mistral-large-latest", messages)` | -| Mistral 7B | `completion(model="mistral/open-mistral-7b", messages)` | -| Mixtral 8x7B | `completion(model="mistral/open-mixtral-8x7b", messages)` | -| Mixtral 8x22B | `completion(model="mistral/open-mixtral-8x22b", messages)` | -| Codestral | `completion(model="mistral/codestral-latest", messages)` | -| Mistral NeMo | `completion(model="mistral/open-mistral-nemo", messages)` | -| Mistral NeMo 2407 | `completion(model="mistral/open-mistral-nemo-2407", messages)` | -| Codestral Mamba | `completion(model="mistral/open-codestral-mamba", messages)` | -| Codestral Mamba | `completion(model="mistral/codestral-mamba-latest"", messages)` | +| Model Name | Function Call | Reasoning Support | +|----------------|--------------------------------------------------------------|-------------------| +| Mistral Small | `completion(model="mistral/mistral-small-latest", messages)` | ❌ | +| Mistral Medium | `completion(model="mistral/mistral-medium-latest", messages)`| ❌ | +| Mistral Large 2 | `completion(model="mistral/mistral-large-2407", messages)` | ❌ | +| Mistral Large Latest | `completion(model="mistral/mistral-large-latest", messages)` | ❌ | +| **Magistral Small** | `completion(model="mistral/magistral-small-2506", messages)` | ✅ | +| **Magistral Medium** | `completion(model="mistral/magistral-medium-2506", messages)`| ✅ | +| Mistral 7B | `completion(model="mistral/open-mistral-7b", messages)` | ❌ | +| Mixtral 8x7B | `completion(model="mistral/open-mixtral-8x7b", messages)` | ❌ | +| Mixtral 8x22B | `completion(model="mistral/open-mixtral-8x22b", messages)` | ❌ | +| Codestral | `completion(model="mistral/codestral-latest", messages)` | ❌ | +| Mistral NeMo | `completion(model="mistral/open-mistral-nemo", messages)` | ❌ | +| Mistral NeMo 2407 | `completion(model="mistral/open-mistral-nemo-2407", messages)` | ❌ | +| Codestral Mamba | `completion(model="mistral/open-codestral-mamba", messages)` | ❌ | +| Codestral Mamba | `completion(model="mistral/codestral-mamba-latest"", messages)` | ❌ | ## Function Calling @@ -203,6 +205,110 @@ assert isinstance( ) ``` +## Reasoning Capabilities (Magistral Models) + +Mistral's Magistral models support advanced reasoning capabilities that allow the model to think step-by-step before providing answers. LiteLLM provides seamless integration with these reasoning features through OpenAI-compatible parameters. + +### Supported Magistral Models + +| Model Name | Function Call | +|----------------|--------------------------------------------------------------| +| Magistral Small | `completion(model="mistral/magistral-small-2506", messages)` | +| Magistral Medium | `completion(model="mistral/magistral-medium-2506", messages)`| + +### Using Reasoning Effort + +The `reasoning_effort` parameter controls how much effort the model puts into reasoning. When used with magistral models. + +```python +from litellm import completion +import os + +os.environ['MISTRAL_API_KEY'] = "your-api-key" + +response = completion( + model="mistral/magistral-medium-2506", + messages=[ + {"role": "user", "content": "What is 15 multiplied by 7?"} + ], + reasoning_effort="medium" # Options: "low", "medium", "high" +) + +print(response) +``` + +### Example with System Message + +If you already have a system message, LiteLLM will prepend the reasoning instructions: + +```python +response = completion( + model="mistral/magistral-medium-2506", + messages=[ + {"role": "system", "content": "You are a helpful math tutor."}, + {"role": "user", "content": "Explain how to solve quadratic equations."} + ], + reasoning_effort="high" +) + +# The system message becomes: +# "When solving problems, think step-by-step in tags before providing your final answer... +# +# You are a helpful math tutor." +``` + +### Usage with LiteLLM Proxy + +You can also use reasoning capabilities through the LiteLLM proxy: + + + + +```shell +curl --location 'http://0.0.0.0:4000/chat/completions' \ +--header 'Content-Type: application/json' \ +--data '{ + "model": "magistral-medium-2506", + "messages": [ + { + "role": "user", + "content": "What is the square root of 144? Show your reasoning." + } + ], + "reasoning_effort": "medium" + }' +``` + + + +```python +import openai +client = openai.OpenAI( + api_key="anything", + base_url="http://0.0.0.0:4000" +) + +response = client.chat.completions.create( + model="magistral-medium-2506", + messages=[ + { + "role": "user", + "content": "Calculate the area of a circle with radius 5. Show your work." + } + ], + reasoning_effort="high" +) + +print(response) +``` + + + +### Important Notes + +- **Model Compatibility**: Reasoning parameters only work with magistral models +- **Backward Compatibility**: Non-magistral models will ignore reasoning parameters and work normally + ## Sample Usage - Embedding ```python from litellm import embedding diff --git a/docs/my-website/docs/reasoning_content.md b/docs/my-website/docs/reasoning_content.md index 7576e34ee35..fb3640fdc11 100644 --- a/docs/my-website/docs/reasoning_content.md +++ b/docs/my-website/docs/reasoning_content.md @@ -19,6 +19,7 @@ Supported Providers: - Google AI Studio (`google/`) - Vertex AI (`vertex_ai/`) - Perplexity (`perplexity/`) +- Mistral AI (Magistral models) (`mistral/`) LiteLLM will standardize the `reasoning_content` in the response and `thinking_blocks` in the assistant message. @@ -39,7 +40,7 @@ LiteLLM will standardize the `reasoning_content` in the response and `thinking_b ## Quick Start - + ```python showLineNumbers from litellm import completion @@ -57,6 +58,25 @@ response = completion( print(response.choices[0].message.content) ``` + + + +```python showLineNumbers +from litellm import completion +import os + +os.environ["MISTRAL_API_KEY"] = "" + +response = completion( + model="mistral/magistral-medium-2506", + messages=[ + {"role": "user", "content": "What is 15 multiplied by 7? Show your reasoning."}, + ], + reasoning_effort="medium", +) +print(response.choices[0].message.content) +``` +