scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 84

index.md updates

Browse source
This commit is contained in:
ishaan-jaff 2023-09-23 12:30:39 -07:00
parent 8198d4f520
commit 712e6038ae
1 changed files with 23 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
docs/my-website/docs/index.md
View file

@ -120,8 +120,6 @@ response = completion(
</Tabs>
### [Supported Providers 100+ LLMs](./providers)
## Streaming
Set `stream=True` in the `completion` args.
<Tabs>
@ -254,7 +252,29 @@ except OpenAIError as e:
print(e)
```
## Calculate Costs & Usage
## Calculate Costs, Usage, Latency
Pass the completion response to `litellm.completion_cost(completion_response=response)` and get the cost
```python
from litellm import completion, completion_cost
import os
os.environ["OPENAI_API_KEY"] = "sk-litellm-7_NPZhMGxY2GoHC59LgbDw"
response = completion(
model="gpt-3.5-turbo",
messages=[{ "content": "Hello, how are you?","role": "user"}]
)
cost = completion_cost(completion_response=response)
print("Cost for completion call with gpt-3.5-turbo: ", f"${float(cost):.10f}")
```
**Output**
```shell
Cost for completion call with gpt-3.5-turbo: $0.0000775000
```
## Caching with LiteLLM