From 7967f2a82ec45cb3265cf36461132a51d22795b2 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Thu, 19 Oct 2023 09:33:51 -0700 Subject: [PATCH] docs(proxy_server.md): clean up docs --- docs/my-website/docs/proxy_server.md | 42 +--------------------------- 1 file changed, 1 insertion(+), 41 deletions(-) diff --git a/docs/my-website/docs/proxy_server.md b/docs/my-website/docs/proxy_server.md index 06d8ab3c4a1..774f2289dca 100644 --- a/docs/my-website/docs/proxy_server.md +++ b/docs/my-website/docs/proxy_server.md @@ -380,8 +380,7 @@ from langroid.language_models.openai_gpt import OpenAIGPTConfig, OpenAIGPT # configure the LLM my_llm_config = OpenAIGPTConfig( #format: "local/[URL where LiteLLM proxy is listening] - chat_model="local/localhost:8000", - chat_context_length=2048, # adjust based on model + api_base="http://0.0.0.0:8000", ) # create llm, one-off interaction @@ -707,45 +706,6 @@ This will return the most recent log (the call that went to the LLM API + the re All logs are saved to a file called `api_logs.json` in the current directory. -### Deploy Proxy - - - - -**Step 1: Clone the repo** -```shell -git clone https://github.com/BerriAI/litellm.git -``` - -**Step 2: Modify `secrets_template.toml`** -Add your api keys / configure default model: -```shell -[keys] -OPENAI_API_KEY="sk-..." - -[general] -default_model = "gpt-3.5-turbo" -``` - -**Step 3: Deploy Proxy** -```shell -docker build -t litellm . && docker run -p 8000:8000 litellm -``` - - -Use this to deploy local models with Ollama that's OpenAI-compatible. - -It works for models like Mistral, Llama2, CodeLlama, etc. (any model supported by [Ollama](https://ollama.ai/library)) - -**usage** -```shell -docker run --name ollama litellm/ollama -``` - -More details 👉 https://hub.docker.com/r/litellm/ollama - - - ### Configure Proxy If you need to: