From 9c97e1513ce5d462ae98a434d1124974a4ef4acd Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Thu, 21 Sep 2023 17:52:52 -0700 Subject: [PATCH] update config --- docs/my-website/docs/completion/config.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/my-website/docs/completion/config.md b/docs/my-website/docs/completion/config.md index 72ccf8e7261..2a4380c4412 100644 --- a/docs/my-website/docs/completion/config.md +++ b/docs/my-website/docs/completion/config.md @@ -40,6 +40,8 @@ response = completion_with_config(model="gpt-3.5-turbo", messages=messages, conf You can also use model configs to automatically select a model based on the prompt size. It checks the number of tokens in the prompt and max tokens for each model. It selects the model with max tokens > prompt tokens. +If the prompt is larger than any available model, it'll automatically trim the prompt (from the middle + preserve any system message), and fit it to the largest model available. + ```python from litellm import completion_with_config import os