scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

(docs) simple proxy

Browse source
This commit is contained in:
ishaan-jaff 2023-11-09 10:46:15 -08:00
parent 86ef2a02f7
commit fbebb28970
1 changed files with 13 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
docs/my-website/docs/simple_proxy.md
View file

@ -342,6 +342,19 @@ print(result)
</Tabs>
## Proxy Configs
The Config allows you to set a `model_list` and define litellm specific params
Supported params in the config
```yaml
model_list:
- model_name: zephyr-alpha
litellm_params: # params for litellm.completion() - https://docs.litellm.ai/docs/completion/input#input---request-body
model: huggingface/HuggingFaceH4/zephyr-7b-alpha
api_base: http://0.0.0.0:8001
- model_name: zephyr-beta
litellm_params:
model: huggingface/HuggingFaceH4/zephyr-7b-beta
api_base: https://<my-hosted-endpoint>
```
### Set Custom Prompt Templates