mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
(docs) proxy
This commit is contained in:
parent
813da8ec04
commit
4c31045cee
1 changed files with 4 additions and 7 deletions
|
|
@ -13,17 +13,14 @@ LiteLLM Server manages:
|
|||
|
||||
```shell
|
||||
$ litellm --model huggingface/bigcode/starcoder
|
||||
|
||||
#INFO: Proxy running on http://0.0.0.0:8000
|
||||
```
|
||||
|
||||
### Test
|
||||
In a new shell, run, this will make an `openai.ChatCompletion` request
|
||||
```shell
|
||||
curl http://0.0.0.0:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"messages": [{"role": "user", "content": "Say this is a test!"}],
|
||||
"temperature": 0.7
|
||||
}'
|
||||
litellm --test
|
||||
```
|
||||
|
||||
This will now automatically route any requests for gpt-3.5-turbo to bigcode starcoder, hosted on huggingface inference endpoints.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue