scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

Update README.md

Browse source
This commit is contained in:
Krish Dholakia 2023-10-26 16:15:43 -07:00 • committed by GitHub
parent 3563ae81a8
commit 04b0d5704a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 7 additions and 22 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
litellm_server/README.md
View file

@ -14,20 +14,11 @@ A simple, fast, and lightweight **OpenAI-compatible server** to call 100+ LLM AP
## Usage
```shell
docker run -e PORT=8000 -p 8000:8000 ghcr.io/berriai/litellm:latest
docker run -e PORT=8000 -e OPENAI_API_KEY=<your-openai-key> -p 8000:8000 ghcr.io/berriai/litellm:latest
# UVICORN: OpenAI Proxy running on http://0.0.0.0:8000
```
## Endpoints:
- `/chat/completions` - chat completions endpoint to call 100+ LLMs
- `/router/completions` - for multiple deployments of the same model (e.g. Azure OpenAI), uses the least used deployment. [Learn more](https://docs.litellm.ai/docs/routing)
- `/models` - available models on server
## Making Requests to Proxy
### Curl
**Call OpenAI**
```shell
curl http://0.0.0.0:8000/v1/chat/completions \
-H "Content-Type: application/json" \
@ -37,16 +28,12 @@ curl http://0.0.0.0:8000/v1/chat/completions \
"temperature": 0.7
}'
```
**Call Bedrock**
```shell
curl http://0.0.0.0:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "bedrock/anthropic.claude-instant-v1",
"messages": [{"role": "user", "content": "Say this is a test!"}],
"temperature": 0.7
}'
```
[**See how to call Huggingface,Bedrock,TogetherAI,Anthropic, etc.**]([https://docs.litellm.ai/docs/simple_proxy](https://docs.litellm.ai/docs/providers))
## Endpoints:
- `/chat/completions` - chat completions endpoint to call 100+ LLMs
- `/router/completions` - for multiple deployments of the same model (e.g. Azure OpenAI), uses the least used deployment. [Learn more](https://docs.litellm.ai/docs/routing)
- `/models` - available models on server
### Running Locally
```shell
@ -59,5 +46,3 @@ $ cd ./litellm/litellm_server
```shell
$ uvicorn main:app --host 0.0.0.0 --port 8000
```
[**See how to call Huggingface,Bedrock,TogetherAI,Anthropic, etc.**](https://docs.litellm.ai/docs/simple_proxy)