scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

Improving the proxy docs for configuring with vllm

Browse source
This commit is contained in:
Keith Stevens 2024-08-13 16:07:41 -07:00
parent a9a3282031
commit 17c6a4e532
2 changed files with 4 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
docs/my-website/docs/proxy/configs.md
View file

@ -55,7 +55,8 @@ model_list:
- model_name: vllm-models
litellm_params:
model: openai/facebook/opt-125m # the `openai/` prefix tells litellm it's openai compatible
api_base: http://0.0.0.0:4000
api_base: http://0.0.0.0:4000/v1
api_key: none
rpm: 1440
model_info:
version: 2

3
docs/my-website/docs/proxy/quick_start.md
View file

@ -243,7 +243,8 @@ model_list:
- model_name: vllm-model
litellm_params:
model: openai/<your-model-name>
api_base: <your-api-base> # e.g. http://0.0.0.0:3000
api_base: <your-vllm-api-base> # e.g. http://0.0.0.0:3000/v1
api_key: <your-vllm-api-key|none>
```
### Run proxy with config