From 7ae7e95da174523ca56d4c932f32582f92faf10b Mon Sep 17 00:00:00 2001 From: ishaan-jaff Date: Sat, 9 Mar 2024 13:04:52 -0800 Subject: [PATCH] (docs) litellm getting started clarify sdk vs proxy --- docs/my-website/docs/index.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/my-website/docs/index.md b/docs/my-website/docs/index.md index d7ed1401950..d3284e6fb95 100644 --- a/docs/my-website/docs/index.md +++ b/docs/my-website/docs/index.md @@ -13,7 +13,14 @@ https://github.com/BerriAI/litellm - Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - [Router](https://docs.litellm.ai/docs/routing) - Track spend & set budgets per project [OpenAI Proxy Server](https://docs.litellm.ai/docs/simple_proxy) -## Basic usage +## How to use LiteLLM +You can use litellm through either: +1. [OpenAI proxy Server](#openai-proxy) - Server to call 100+ LLMs, load balance, cost tracking across projects +2. [LiteLLM python SDK](#basic-usage) - Python Client to call 100+ LLMs, load balance, cost tracking + +## LiteLLM Python SDK + +### Basic usage Open In Colab @@ -144,7 +151,7 @@ response = completion( -## Streaming +### Streaming Set `stream=True` in the `completion` args. @@ -276,7 +283,7 @@ response = completion( -## Exception handling +### Exception handling LiteLLM maps exceptions across all supported providers to the OpenAI exceptions. All our exceptions inherit from OpenAI's exception types, so any error-handling you have for that, should work out of the box with LiteLLM. @@ -292,7 +299,7 @@ except OpenAIError as e: print(e) ``` -## Logging Observability - Log LLM Input/Output ([Docs](https://docs.litellm.ai/docs/observability/callbacks)) +### Logging Observability - Log LLM Input/Output ([Docs](https://docs.litellm.ai/docs/observability/callbacks)) LiteLLM exposes pre defined callbacks to send data to Langfuse, LLMonitor, Helicone, Promptlayer, Traceloop, Slack ```python from litellm import completion @@ -311,7 +318,7 @@ litellm.success_callback = ["langfuse", "llmonitor"] # log input/output to langf response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}]) ``` -## Track Costs, Usage, Latency for streaming +### Track Costs, Usage, Latency for streaming Use a callback function for this - more info on custom callbacks: https://docs.litellm.ai/docs/observability/custom_callback ```python