scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102

Merge pull request #5003 from BerriAI/litellm_add_db_llama

Browse source 
Add `databricks/databricks-meta-llama-3-1-70b-instruct`, 'databricks/databricks-meta-llama-3-1-405b-instruct'
This commit is contained in:
Ishaan Jaff 2024-08-01 17:11:26 -07:00 • committed by GitHub
commit fc64c957cf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 63 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

21
docs/my-website/docs/providers/databricks.md
View file

@ -5,6 +5,11 @@ import TabItem from '@theme/TabItem';
LiteLLM supports all models on Databricks
:::tip
**We support ALL Databricks models, just set `model=databricks/<any-model-on-databricks>` as a prefix when sending litellm requests**
:::
## Usage
@ -185,8 +190,17 @@ response = litellm.embedding(
## Supported Databricks Chat Completion Models
:::tip
**We support ALL Databricks models, just set `model=databricks/<any-model-on-databricks>` as a prefix when sending litellm requests**
:::
| Model Name | Command |
|----------------------------|------------------------------------------------------------------|
| databricks-meta-llama-3-1-70b-instruct | `completion(model='databricks/databricks-meta-llama-3-1-70b-instruct', messages=messages)` |
| databricks-meta-llama-3-1-405b-instruct | `completion(model='databricks/databricks-meta-llama-3-1-405b-instruct', messages=messages)` |
| databricks-dbrx-instruct | `completion(model='databricks/databricks-dbrx-instruct', messages=messages)` |
| databricks-meta-llama-3-70b-instruct | `completion(model='databricks/databricks-meta-llama-3-70b-instruct', messages=messages)` |
| databricks-llama-2-70b-chat | `completion(model='databricks/databricks-llama-2-70b-chat', messages=messages)` |
@ -196,6 +210,13 @@ response = litellm.embedding(
## Supported Databricks Embedding Models
:::tip
**We support ALL Databricks models, just set `model=databricks/<any-model-on-databricks>` as a prefix when sending litellm requests**
:::
| Model Name | Command |
|----------------------------|------------------------------------------------------------------|
| databricks-bge-large-en | `embedding(model='databricks/databricks-bge-large-en', messages=messages)` |

22
litellm/model_prices_and_context_window_backup.json
View file

@ -4698,6 +4698,26 @@
"litellm_provider": "voyage",
"mode": "embedding"
},
"databricks/databricks-meta-llama-3-1-405b-instruct": {
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.000005,
"output_cost_per_token": 0.000015,
"litellm_provider": "databricks",
"mode": "chat",
"source": "https://www.databricks.com/product/pricing/foundation-model-serving"
},
"databricks/databricks-meta-llama-3-1-70b-instruct": {
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.000001,
"output_cost_per_token": 0.000003,
"litellm_provider": "databricks",
"mode": "chat",
"source": "https://www.databricks.com/product/pricing/foundation-model-serving"
},
"databricks/databricks-dbrx-instruct": {
"max_tokens": 32768,
"max_input_tokens": 32768,
@ -4768,5 +4788,5 @@
"litellm_provider": "databricks",
"mode": "embedding",
"source": "https://www.databricks.com/product/pricing/foundation-model-serving"
}
},
}

22
model_prices_and_context_window.json
View file

@ -4698,6 +4698,26 @@
"litellm_provider": "voyage",
"mode": "embedding"
},
"databricks/databricks-meta-llama-3-1-405b-instruct": {
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.000005,
"output_cost_per_token": 0.000015,
"litellm_provider": "databricks",
"mode": "chat",
"source": "https://www.databricks.com/product/pricing/foundation-model-serving"
},
"databricks/databricks-meta-llama-3-1-70b-instruct": {
"max_tokens": 128000,
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"input_cost_per_token": 0.000001,
"output_cost_per_token": 0.000003,
"litellm_provider": "databricks",
"mode": "chat",
"source": "https://www.databricks.com/product/pricing/foundation-model-serving"
},
"databricks/databricks-dbrx-instruct": {
"max_tokens": 32768,
"max_input_tokens": 32768,
@ -4768,5 +4788,5 @@
"litellm_provider": "databricks",
"mode": "embedding",
"source": "https://www.databricks.com/product/pricing/foundation-model-serving"
}
},
}