scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

(docs) add cost tracking to proxy server

Browse source
This commit is contained in:
ishaan-jaff 2023-10-10 10:19:15 -07:00
parent af2fd0e0de
commit cf580a4995
1 changed files with 16 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
docs/my-website/docs/proxy_server.md
View file

@ -248,7 +248,22 @@ response = openai.ChatCompletion.create(model="ollama/llama2", messages=[{"role"
print(response)
```
### Tracking costs
By default litellm proxy writes cost logs to litellm/proxy/cost.log
```shell
2023-10-09 14:48:18 - Model gpt-3.5-turbo-0613 Cost: 0.000047000
2023-10-09 14:48:18 - Model gpt-3.5-turbo-0613 Cost: 0.000128700
2023-10-09 14:48:20 - Model gpt-3.5-turbo-0613 Cost: 0.000523011
2023-10-09 14:48:28 - Model ollama/llama2 Cost: 0.00004700
2023-10-09 14:48:35 - Model ollama/code-llama Cost: 0.00004700
2023-10-09 14:48:49 - Model azure/chatgpt-2 Cost: 0.000223011
```
You can view costs on the cli using
``shell
litellm --cost
```
### Ollama Logs
Ollama calls can sometimes fail (out-of-memory errors, etc.).