mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
docs(input.md): add litellm params to docs
This commit is contained in:
Krrish Dholakia
parent
758b75cb8d
commit
585f7b0434
2 changed files with 90 additions and 57 deletions
|
|
@ -58,6 +58,96 @@ To drop the param instead, set `litellm.drop_params = True`.
|
|||
Add to prompt for non-openai models, set: `litellm.add_function_to_prompt = True`.
|
||||
:::
|
||||
|
||||
## Input Params
|
||||
|
||||
```python
|
||||
def completion(
|
||||
model: str,
|
||||
messages: List = [],
|
||||
# Optional OpenAI params
|
||||
functions: List = [],
|
||||
function_call: str = "", # optional params
|
||||
temperature: Optional[float] = None,
|
||||
top_p: Optional[float] = None,
|
||||
n: Optional[int] = None,
|
||||
stream: Optional[bool] = None,
|
||||
stop=None,
|
||||
max_tokens: Optional[float] = None,
|
||||
presence_penalty: Optional[float] = None,
|
||||
frequency_penalty: Optional[float]=None,
|
||||
logit_bias: dict = {},
|
||||
user: str = "",
|
||||
deployment_id = None,
|
||||
request_timeout: Optional[int] = None,
|
||||
|
||||
# Optional LiteLLM params
|
||||
api_base: Optional[str] = None,
|
||||
api_version: Optional[str] = None,
|
||||
api_key: Optional[str] = None,
|
||||
fallbacks: Optional[list] = None, # pass in a list of api_base,keys, etc.
|
||||
|
||||
**kwargs,
|
||||
) -> ModelResponse:
|
||||
```
|
||||
### Required Fields
|
||||
|
||||
- `model`: *string* - ID of the model to use. Refer to the model endpoint compatibility table for details on which models work with the Chat API.
|
||||
|
||||
- `messages`: *array* - A list of messages comprising the conversation so far.
|
||||
|
||||
#### Properties of `messages`
|
||||
*Note* - Each message in the array contains the following properties:
|
||||
|
||||
- `role`: *string* - The role of the message's author. Roles can be: system, user, assistant, or function.
|
||||
|
||||
- `content`: *string or null* - The contents of the message. It is required for all messages, but may be null for assistant messages with function calls.
|
||||
|
||||
- `name`: *string (optional)* - The name of the author of the message. It is required if the role is "function". The name should match the name of the function represented in the content. It can contain characters (a-z, A-Z, 0-9), and underscores, with a maximum length of 64 characters.
|
||||
|
||||
- `function_call`: *object (optional)* - The name and arguments of a function that should be called, as generated by the model.
|
||||
|
||||
|
||||
|
||||
### Optional Fields
|
||||
|
||||
- `functions`: *array* - A list of functions that the model may use to generate JSON inputs. Each function should have the following properties:
|
||||
|
||||
- `name`: *string* - The name of the function to be called. It should contain a-z, A-Z, 0-9, underscores and dashes, with a maximum length of 64 characters.
|
||||
|
||||
- `description`: *string (optional)* - A description explaining what the function does. It helps the model to decide when and how to call the function.
|
||||
|
||||
- `parameters`: *object* - The parameters that the function accepts, described as a JSON Schema object.
|
||||
|
||||
- `function_call`: *string or object (optional)* - Controls how the model responds to function calls.
|
||||
|
||||
- `temperature`: *number or null (optional)* - The sampling temperature to be used, between 0 and 2. Higher values like 0.8 produce more random outputs, while lower values like 0.2 make outputs more focused and deterministic.
|
||||
|
||||
- `top_p`: *number or null (optional)* - An alternative to sampling with temperature. It instructs the model to consider the results of the tokens with top_p probability. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered.
|
||||
|
||||
- `n`: *integer or null (optional)* - The number of chat completion choices to generate for each input message.
|
||||
|
||||
- `stream`: *boolean or null (optional)* - If set to true, it sends partial message deltas. Tokens will be sent as they become available, with the stream terminated by a [DONE] message.
|
||||
|
||||
- `stop`: *string/ array/ null (optional)* - Up to 4 sequences where the API will stop generating further tokens.
|
||||
|
||||
- `max_tokens`: *integer (optional)* - The maximum number of tokens to generate in the chat completion.
|
||||
|
||||
- `presence_penalty`: *number or null (optional)* - It is used to penalize new tokens based on their existence in the text so far.
|
||||
|
||||
- `frequency_penalty`: *number or null (optional)* - It is used to penalize new tokens based on their frequency in the text so far.
|
||||
|
||||
- `logit_bias`: *map (optional)* - Used to modify the probability of specific tokens appearing in the completion.
|
||||
|
||||
- `user`: *string (optional)* - A unique identifier representing your end-user. This can help OpenAI to monitor and detect abuse.
|
||||
|
||||
- `request_timeout`: *int (optional)* - Timeout in seconds for completion requests (Defaults to 600 seconds)
|
||||
|
||||
- `api_base`: *string (optional)* - The api endpoint you want to call the model with
|
||||
|
||||
- `api_version`: *string (optional)* - (Azure-specific) the api version for the call
|
||||
|
||||
- `fallbacks`: *list (optional)* - A list of model names + params to be used, in case the initial call fails
|
||||
|
||||
## Provider-specific Params
|
||||
Providers might offer params not supported by OpenAI (e.g. top_k). You can pass those in 2 ways:
|
||||
- via completion(): We'll pass the non-openai param, straight to the provider as part of the request body.
|
||||
|
|
@ -453,59 +543,3 @@ assert len(response_2_text) > len(response_1_text)
|
|||
|
||||
|
||||
[**Check out the tutorial!**](../tutorials/provider_specific_params.md)
|
||||
|
||||
## Input - Request Body
|
||||
# Request Body
|
||||
|
||||
### Required Fields
|
||||
|
||||
- `model`: *string* - ID of the model to use. Refer to the model endpoint compatibility table for details on which models work with the Chat API.
|
||||
|
||||
- `messages`: *array* - A list of messages comprising the conversation so far.
|
||||
|
||||
#### Properties of `messages`
|
||||
*Note* - Each message in the array contains the following properties:
|
||||
|
||||
- `role`: *string* - The role of the message's author. Roles can be: system, user, assistant, or function.
|
||||
|
||||
- `content`: *string or null* - The contents of the message. It is required for all messages, but may be null for assistant messages with function calls.
|
||||
|
||||
- `name`: *string (optional)* - The name of the author of the message. It is required if the role is "function". The name should match the name of the function represented in the content. It can contain characters (a-z, A-Z, 0-9), and underscores, with a maximum length of 64 characters.
|
||||
|
||||
- `function_call`: *object (optional)* - The name and arguments of a function that should be called, as generated by the model.
|
||||
|
||||
|
||||
|
||||
### Optional Fields
|
||||
|
||||
- `functions`: *array* - A list of functions that the model may use to generate JSON inputs. Each function should have the following properties:
|
||||
|
||||
- `name`: *string* - The name of the function to be called. It should contain a-z, A-Z, 0-9, underscores and dashes, with a maximum length of 64 characters.
|
||||
|
||||
- `description`: *string (optional)* - A description explaining what the function does. It helps the model to decide when and how to call the function.
|
||||
|
||||
- `parameters`: *object* - The parameters that the function accepts, described as a JSON Schema object.
|
||||
|
||||
- `function_call`: *string or object (optional)* - Controls how the model responds to function calls.
|
||||
|
||||
- `temperature`: *number or null (optional)* - The sampling temperature to be used, between 0 and 2. Higher values like 0.8 produce more random outputs, while lower values like 0.2 make outputs more focused and deterministic.
|
||||
|
||||
- `top_p`: *number or null (optional)* - An alternative to sampling with temperature. It instructs the model to consider the results of the tokens with top_p probability. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered.
|
||||
|
||||
- `n`: *integer or null (optional)* - The number of chat completion choices to generate for each input message.
|
||||
|
||||
- `stream`: *boolean or null (optional)* - If set to true, it sends partial message deltas. Tokens will be sent as they become available, with the stream terminated by a [DONE] message.
|
||||
|
||||
- `stop`: *string/ array/ null (optional)* - Up to 4 sequences where the API will stop generating further tokens.
|
||||
|
||||
- `max_tokens`: *integer (optional)* - The maximum number of tokens to generate in the chat completion.
|
||||
|
||||
- `presence_penalty`: *number or null (optional)* - It is used to penalize new tokens based on their existence in the text so far.
|
||||
|
||||
- `frequency_penalty`: *number or null (optional)* - It is used to penalize new tokens based on their frequency in the text so far.
|
||||
|
||||
- `logit_bias`: *map (optional)* - Used to modify the probability of specific tokens appearing in the completion.
|
||||
|
||||
- `user`: *string (optional)* - A unique identifier representing your end-user. This can help OpenAI to monitor and detect abuse.
|
||||
|
||||
- `request_timeout`: *int (optional)* - Timeout in seconds for completion requests (Defaults to 600 seconds)
|
||||
|
|
@ -37,7 +37,6 @@ const sidebars = {
|
|||
"completion/function_call",
|
||||
"completion/model_alias",
|
||||
"completion/reliable_completions",
|
||||
"completion/multiple_deployments",
|
||||
"completion/config",
|
||||
"completion/batching",
|
||||
"completion/mock_requests",
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue