From 27c045fed093d4f752fa2151ee7219743af10c15 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Wed, 19 Jun 2024 13:12:22 -0700 Subject: [PATCH] docs(health.md): add doc on health checking specific models --- docs/my-website/docs/proxy/health.md | 42 ++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/docs/my-website/docs/proxy/health.md b/docs/my-website/docs/proxy/health.md index 03dd917315c..c67302e0d0d 100644 --- a/docs/my-website/docs/proxy/health.md +++ b/docs/my-website/docs/proxy/health.md @@ -161,4 +161,46 @@ Example Response: ```json "I'm alive!" +``` + +## Advanced - Call specific models + +To check health of specific models, here's how to call them: + +### 1. Get model id via `/model/info` + +```bash +curl -X GET 'http://0.0.0.0:4000/v1/model/info' \ +--header 'Authorization: Bearer sk-1234' \ +``` + +**Expected Response** + +```bash +{ + "model_name": "bedrock-anthropic-claude-3", + "litellm_params": { + "model": "anthropic.claude-3-sonnet-20240229-v1:0" + }, + "model_info": { + "id": "634b87c444..", # 👈 UNIQUE MODEL ID +} +``` + +### 2. Call specific model via `/chat/completions` + +```bash +curl -X POST 'http://localhost:4000/chat/completions' \ +-H 'Content-Type: application/json' \ +-H 'Authorization: Bearer sk-1234' \ +-D '{ + "model": "634b87c444.." # 👈 UNIQUE MODEL ID + "messages": [ + { + "role": "user", + "content": "ping" + } + ], +} +' ``` \ No newline at end of file