scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 129

docs(completion-docs): add ollama provider specific settings

Browse source
This commit is contained in:
Krrish Dholakia 2023-10-07 13:53:01 -07:00
parent d69038883c
commit 78e413675e
1 changed files with 29 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/my-website/docs/completion/input.md
View file

@ -258,6 +258,35 @@ assert len(response_2_text) > len(response_1_text)
</TabItem>
<TabItem value="ollama" label="Ollama">
```python
import litellm, os
## SET MAX TOKENS - via completion()
response_1 = litellm.completion(
model="ollama/llama2",
messages=[{ "content": "Hello, how are you?","role": "user"}],
max_tokens=10
)
response_1_text = response_1.choices[0].message.content
## SET MAX TOKENS - via config
litellm.OllamConfig(num_predict=200)
response_2 = litellm.completion(
model="ollama/llama2",
messages=[{ "content": "Hello, how are you?","role": "user"}],
)
response_2_text = response_2.choices[0].message.content
## TEST OUTPUT
assert len(response_2_text) > len(response_1_text)
```
</TabItem>
<TabItem value="replicate" label="Replicate">
```python