From dbc7552d15949f4af684f8127550992934e75845 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Wed, 13 Mar 2024 12:26:39 -0700 Subject: [PATCH] docs: refactor team based logging in docs --- docs/my-website/docs/proxy/logging.md | 28 ----------- .../docs/proxy/team_based_routing.md | 46 ++++++++++++++++--- 2 files changed, 40 insertions(+), 34 deletions(-) diff --git a/docs/my-website/docs/proxy/logging.md b/docs/my-website/docs/proxy/logging.md index 589199a07f4..bdd75d647cc 100644 --- a/docs/my-website/docs/proxy/logging.md +++ b/docs/my-website/docs/proxy/logging.md @@ -678,34 +678,6 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \ Your logs should be available on the specified s3 Bucket -## Team-based Logging - -Set success callbacks (e.g. langfuse), for a specific team-id. - -```yaml -litellm_settings: - default_team_settings: - - team_id: my-secret-project - success_callback: ["langfuse"] - langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_2 - langfuse_secret: os.environ/LANGFUSE_PRIVATE_KEY_2 - - team_id: ishaans-secret-project - success_callback: ["langfuse"] - langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_3 - langfuse_secret: os.environ/LANGFUSE_SECRET_3 -``` - -Now, when you [generate keys](./virtual_keys.md) for this team-id - -```bash -curl -X POST 'http://0.0.0.0:4000/key/generate' \ --H 'Authorization: Bearer sk-1234' \ --H 'Content-Type: application/json' \ --D '{"team_id": "ishaans-secret-project"}' -``` - -All requests made with these keys will log data to their team-specific logging. - ## Logging Proxy Input/Output - DynamoDB We will use the `--config` to set diff --git a/docs/my-website/docs/proxy/team_based_routing.md b/docs/my-website/docs/proxy/team_based_routing.md index 4b057cdf382..4f0b7a2ae3a 100644 --- a/docs/my-website/docs/proxy/team_based_routing.md +++ b/docs/my-website/docs/proxy/team_based_routing.md @@ -1,8 +1,9 @@ -# 👥 Team-based Routing +# 👥 Team-based Routing + Logging +## Routing Route calls to different model groups based on the team-id -## Config with model group +### Config with model group Create a config.yaml with 2 model groups + connected postgres db @@ -32,7 +33,7 @@ Start proxy litellm --config /path/to/config.yaml ``` -## Create Team with Model Alias +### Create Team with Model Alias ```bash curl --location 'http://0.0.0.0:4000/team/new' \ @@ -46,7 +47,7 @@ curl --location 'http://0.0.0.0:4000/team/new' \ # Returns team_id: my-team-id ``` -## Create Team Key +### Create Team Key ```bash curl --location 'http://localhost:4000/key/generate' \ @@ -57,7 +58,7 @@ curl --location 'http://localhost:4000/key/generate' \ }' ``` -## Call Model with alias +### Call Model with alias ```bash curl --location 'http://0.0.0.0:4000/v1/chat/completions' \ @@ -68,4 +69,37 @@ curl --location 'http://0.0.0.0:4000/v1/chat/completions' \ "messages": [{"role": "system", "content": "You'\''re an expert at writing poems"}, {"role": "user", "content": "Write me a poem"}, {"role": "user", "content": "What'\''s your name?"}], "user": "usha" }' -``` \ No newline at end of file +``` + + +## Logging / Caching + +Turn on/off logging and caching for a specific team id. + +**Example:** + +This config would send langfuse logs to 2 different langfuse projects, based on the team id + +```yaml +litellm_settings: + default_team_settings: + - team_id: my-secret-project + success_callback: ["langfuse"] + langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_1 # Project 1 + langfuse_secret: os.environ/LANGFUSE_PRIVATE_KEY_1 # Project 1 + - team_id: ishaans-secret-project + success_callback: ["langfuse"] + langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_2 # Project 2 + langfuse_secret: os.environ/LANGFUSE_SECRET_2 # Project 2 +``` + +Now, when you [generate keys](./virtual_keys.md) for this team-id + +```bash +curl -X POST 'http://0.0.0.0:4000/key/generate' \ +-H 'Authorization: Bearer sk-1234' \ +-H 'Content-Type: application/json' \ +-D '{"team_id": "ishaans-secret-project"}' +``` + +All requests made with these keys will log data to their team-specific logging.