scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-11 22:51:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 154

docs(cost_tracking.md): cleanup docs

Browse source
This commit is contained in:
Krrish Dholakia 2024-03-19 17:40:52 -07:00
parent 191c2256f6
commit 0c1d754e2c
2 changed files with 25 additions and 19 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
docs/my-website/docs/proxy/configs.md
View file

@ -246,6 +246,10 @@ $ litellm --config /path/to/config.yaml
## Load Balancing
:::info
For more on this, go to [this page](./load_balancing.md)
:::
Use this to call multiple instances of the same model and configure things like [routing strategy](../routing.md#advanced).
For optimal performance:
@ -306,25 +310,6 @@ router_settings: # router_settings are optional
redis_port: 1992
```
## Set Azure `base_model` for cost tracking
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
Example config with `base_model`
```yaml
model_list:
- model_name: azure-gpt-3.5
litellm_params:
model: azure/chatgpt-v-2
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: "2023-07-01-preview"
model_info:
base_model: azure/gpt-4-1106-preview
```
You can view your cost once you set up [Virtual keys](https://docs.litellm.ai/docs/proxy/virtual_keys) or [custom_callbacks](https://docs.litellm.ai/docs/proxy/logging)
## Load API Keys

21
docs/my-website/docs/proxy/cost_tracking.md
View file

@ -15,4 +15,25 @@ model_list:
base_model: dall-e-3 # 👈 set dall-e-3 as base model
model_info:
mode: image_generation
```
## Chat Completions / Embeddings
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
Example config with `base_model`
```yaml
model_list:
- model_name: azure-gpt-3.5
litellm_params:
model: azure/chatgpt-v-2
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: "2023-07-01-preview"
model_info:
base_model: azure/gpt-4-1106-preview
```