mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-07 02:59:05 +00:00
Update request tags doc to show model-level config
- Set tags directly on model deployments in litellm_params - Requests just specify model, tags applied automatically - Use clear naming: AWS_IAM_PROD, AWS_IAM_DEV Co-authored-by: ishaan <ishaan@berri.ai>
This commit is contained in:
parent
ef8699a47c
commit
0c65cbf67f
1 changed files with 22 additions and 40 deletions
|
|
@ -1,70 +1,52 @@
|
|||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# Request Tags for Spend Tracking
|
||||
|
||||
Add tags to your LLM requests to track spend by environment, AWS account, team, or any custom label.
|
||||
Add tags to model deployments to track spend by environment, AWS account, or any custom label.
|
||||
|
||||
Tags appear in the `request_tags` field of LiteLLM spend logs.
|
||||
|
||||
## Set Default Tags in Config
|
||||
## Config Setup
|
||||
|
||||
Set default tags for all API keys generated through the proxy:
|
||||
Set tags on model deployments in `config.yaml`:
|
||||
|
||||
```yaml title="config.yaml"
|
||||
litellm_settings:
|
||||
default_key_generate_params:
|
||||
metadata:
|
||||
tags: ["AWS_IAM_PROD", "us-east-1"]
|
||||
model_list:
|
||||
- model_name: gpt-4
|
||||
litellm_params:
|
||||
model: azure/gpt-4-prod
|
||||
api_key: os.environ/AZURE_PROD_API_KEY
|
||||
api_base: https://prod.openai.azure.com/
|
||||
tags: ["AWS_IAM_PROD"] # 👈 Tag for production
|
||||
|
||||
- model_name: gpt-4-dev
|
||||
litellm_params:
|
||||
model: azure/gpt-4-dev
|
||||
api_key: os.environ/AZURE_DEV_API_KEY
|
||||
api_base: https://dev.openai.azure.com/
|
||||
tags: ["AWS_IAM_DEV"] # 👈 Tag for development
|
||||
```
|
||||
|
||||
All keys created via `/key/generate` will automatically include these tags.
|
||||
## Make Request
|
||||
|
||||
## Send Tags in Request
|
||||
|
||||
Pass tags in the request body:
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="curl" label="cURL">
|
||||
Requests just specify the model - tags are automatically applied:
|
||||
|
||||
```bash
|
||||
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
|
||||
-H 'Authorization: Bearer sk-1234' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"metadata": {
|
||||
"tags": ["AWS_IAM_PROD", "us-east-1"]
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="header" label="Header">
|
||||
|
||||
```bash
|
||||
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
|
||||
-H 'Authorization: Bearer sk-1234' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'x-litellm-tags: AWS_IAM_PROD,us-east-1' \
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hello"}]
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
## Spend Logs
|
||||
|
||||
## Where Tags Appear
|
||||
|
||||
Tags are stored in `LiteLLM_SpendLogs` under `request_tags`:
|
||||
The tag from the model config appears in `LiteLLM_SpendLogs`:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "chatcmpl-abc123",
|
||||
"request_tags": ["AWS_IAM_PROD", "us-east-1"],
|
||||
"request_tags": ["AWS_IAM_PROD"],
|
||||
"spend": 0.002,
|
||||
"model": "gpt-4"
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue