From 04b0d5704a30ba022afdab0861fc109cb09d5413 Mon Sep 17 00:00:00 2001 From: Krish Dholakia Date: Thu, 26 Oct 2023 16:15:43 -0700 Subject: [PATCH] Update README.md --- litellm_server/README.md | 29 +++++++---------------------- 1 file changed, 7 insertions(+), 22 deletions(-) diff --git a/litellm_server/README.md b/litellm_server/README.md index 2eb586d220c..09b187c8f34 100644 --- a/litellm_server/README.md +++ b/litellm_server/README.md @@ -14,20 +14,11 @@ A simple, fast, and lightweight **OpenAI-compatible server** to call 100+ LLM AP ## Usage ```shell -docker run -e PORT=8000 -p 8000:8000 ghcr.io/berriai/litellm:latest +docker run -e PORT=8000 -e OPENAI_API_KEY= -p 8000:8000 ghcr.io/berriai/litellm:latest # UVICORN: OpenAI Proxy running on http://0.0.0.0:8000 ``` -## Endpoints: -- `/chat/completions` - chat completions endpoint to call 100+ LLMs -- `/router/completions` - for multiple deployments of the same model (e.g. Azure OpenAI), uses the least used deployment. [Learn more](https://docs.litellm.ai/docs/routing) -- `/models` - available models on server - -## Making Requests to Proxy -### Curl - -**Call OpenAI** ```shell curl http://0.0.0.0:8000/v1/chat/completions \ -H "Content-Type: application/json" \ @@ -37,16 +28,12 @@ curl http://0.0.0.0:8000/v1/chat/completions \ "temperature": 0.7 }' ``` -**Call Bedrock** -```shell -curl http://0.0.0.0:8000/v1/chat/completions \ - -H "Content-Type: application/json" \ - -d '{ - "model": "bedrock/anthropic.claude-instant-v1", - "messages": [{"role": "user", "content": "Say this is a test!"}], - "temperature": 0.7 - }' -``` + +[**See how to call Huggingface,Bedrock,TogetherAI,Anthropic, etc.**]([https://docs.litellm.ai/docs/simple_proxy](https://docs.litellm.ai/docs/providers)) +## Endpoints: +- `/chat/completions` - chat completions endpoint to call 100+ LLMs +- `/router/completions` - for multiple deployments of the same model (e.g. Azure OpenAI), uses the least used deployment. [Learn more](https://docs.litellm.ai/docs/routing) +- `/models` - available models on server ### Running Locally ```shell @@ -59,5 +46,3 @@ $ cd ./litellm/litellm_server ```shell $ uvicorn main:app --host 0.0.0.0 --port 8000 ``` - -[**See how to call Huggingface,Bedrock,TogetherAI,Anthropic, etc.**](https://docs.litellm.ai/docs/simple_proxy)