mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-10 03:28:53 +00:00
doc cleanup
This commit is contained in:
parent
eeeb95a6ac
commit
ac5fcb673b
3 changed files with 26 additions and 41 deletions
|
|
@ -1,34 +0,0 @@
|
|||
# get context window & cost per token
|
||||
|
||||
100+ LLMs supported: See full list [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
|
||||
|
||||
## using api.litellm.ai
|
||||
```shell
|
||||
curl 'https://api.litellm.ai/get_max_tokens?model=claude-2'
|
||||
```
|
||||
|
||||
### output
|
||||
```json
|
||||
{
|
||||
"input_cost_per_token": 1.102e-05,
|
||||
"max_tokens": 100000,
|
||||
"model": "claude-2",
|
||||
"output_cost_per_token": 3.268e-05
|
||||
}
|
||||
```
|
||||
|
||||
## using the litellm python package
|
||||
```python
|
||||
import litellm
|
||||
model_data = litellm.model_cost["gpt-4"]
|
||||
```
|
||||
|
||||
### output
|
||||
```json
|
||||
{
|
||||
"input_cost_per_token": 3e-06,
|
||||
"max_tokens": 8192,
|
||||
"model": "gpt-4",
|
||||
"output_cost_per_token": 6e-05
|
||||
}
|
||||
```
|
||||
|
|
@ -1,17 +1,21 @@
|
|||
# Completion Token Usage & Cost
|
||||
By default LiteLLM returns token usage in all completion requests ([See here](https://litellm.readthedocs.io/en/latest/output/))
|
||||
|
||||
However, we also expose 5 public helper functions to calculate token usage across providers:
|
||||
However, we also expose 5 helper functions + **[NEW]** an API to calculate token usage across providers:
|
||||
|
||||
- `token_counter`: This returns the number of tokens for a given input - it uses the tokenizer based on the model, and defaults to tiktoken if no model-specific tokenizer is available.
|
||||
- `token_counter`: This returns the number of tokens for a given input - it uses the tokenizer based on the model, and defaults to tiktoken if no model-specific tokenizer is available. [**Jump to code**](#1-token_counter)
|
||||
|
||||
- `cost_per_token`: This returns the cost (in USD) for prompt (input) and completion (output) tokens. It utilizes our model_cost map which can be found in `__init__.py` and also as a [community resource](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json).
|
||||
- `cost_per_token`: This returns the cost (in USD) for prompt (input) and completion (output) tokens. Uses the live list from `api.litellm.ai`. [**Jump to code**](#2-cost_per_token)
|
||||
|
||||
- `completion_cost`: This returns the overall cost (in USD) for a given LLM API Call. It combines `token_counter` and `cost_per_token` to return the cost for that query (counting both cost of input and output).
|
||||
- `completion_cost`: This returns the overall cost (in USD) for a given LLM API Call. It combines `token_counter` and `cost_per_token` to return the cost for that query (counting both cost of input and output). [**Jump to code**](#3-completion_cost)
|
||||
|
||||
- `get_max_tokens`: This returns a dictionary for a specific model, with it's max_tokens, input_cost_per_token and output_cost_per_token
|
||||
- `get_max_tokens`: This returns a dictionary for a specific model, with it's max_tokens, input_cost_per_token and output_cost_per_token. [**Jump to code**](#4-get_max_tokens)
|
||||
|
||||
- `model_cost`: This returns a dictionary for all models, with their max_tokens, input_cost_per_token and output_cost_per_token [**List of all models**](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) (📣 This is a community maintained list. Contributions are welcome! ❤️)
|
||||
- `model_cost`: This returns a dictionary for all models, with their max_tokens, input_cost_per_token and output_cost_per_token. It uses the `api.litellm.ai` call shown below. [**Jump to code**](#5-model_cost)
|
||||
|
||||
- `api.litellm.ai`: Live token + price count across [all supported models](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json). [**Jump to code**](#6-apilitellmai)
|
||||
|
||||
📣 This is a community maintained list. Contributions are welcome! ❤️
|
||||
|
||||
## Example Usage
|
||||
|
||||
|
|
@ -78,3 +82,19 @@ from litellm import model_cost
|
|||
print(model_cost) # {'gpt-3.5-turbo': {'max_tokens': 4000, 'input_cost_per_token': 1.5e-06, 'output_cost_per_token': 2e-06}, ...}
|
||||
```
|
||||
|
||||
### 6. `api.litellm.ai`
|
||||
|
||||
**Example Curl Request**
|
||||
```shell
|
||||
curl 'https://api.litellm.ai/get_max_tokens?model=claude-2'
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"input_cost_per_token": 1.102e-05,
|
||||
"max_tokens": 100000,
|
||||
"model": "claude-2",
|
||||
"output_cost_per_token": 3.268e-05
|
||||
}
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -80,7 +80,6 @@ const sidebars = {
|
|||
"token_usage",
|
||||
"exception_mapping",
|
||||
'debugging/local_debugging',
|
||||
"max_tokens_cost",
|
||||
"budget_manager",
|
||||
"proxy_api",
|
||||
{
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue