scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 79

Merge pull request #4533 from UsableMachines/nick/fix-granite-prompt-template

Browse source 
Fix Granite Prompt template
This commit is contained in:
Krish Dholakia 2024-07-03 18:45:25 -07:00 • committed by GitHub
commit 1078c0f77a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 6 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
litellm/llms/prompt_templates/factory.py
View file

@ -531,6 +531,7 @@ def format_prompt_togetherai(messages, prompt_format, chat_template):
### IBM Granite
def ibm_granite_pt(messages: list):
"""
IBM's Granite models uses the template:
@ -547,10 +548,13 @@ def ibm_granite_pt(messages: list):
},
"user": {
"pre_message": "<|user|>\n",
"post_message": "\n",
# Assistant tag is needed in the prompt after the user message
# to avoid the model completing the users sentence before it answers
# https://www.ibm.com/docs/en/watsonx/w-and-w/2.0.x?topic=models-granite-13b-chat-v2-prompting-tips#chat
"post_message": "\n<|assistant|>\n",
},
"assistant": {
"pre_message": "<|assistant|>\n",
"pre_message": "",
"post_message": "\n",
},
},