mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-07 02:59:05 +00:00
(docs) provisioned throughput bedrock models
This commit is contained in:
parent
069d060ec9
commit
7af3fe68e4
1 changed files with 26 additions and 0 deletions
|
|
@ -208,6 +208,32 @@ response = completion(
|
|||
)
|
||||
```
|
||||
|
||||
## Provisioned throughput models
|
||||
To use provisioned throughput Bedrock models pass
|
||||
- `model=bedrock/<base-model>`, example `model=bedrock/anthropic.claude-v2`. Set `model` to any of the [Supported AWS models](#supported-aws-bedrock-models)
|
||||
- `model_id=provisioned-model-arn`
|
||||
|
||||
Completion
|
||||
```python
|
||||
import litellm
|
||||
response = litellm.completion(
|
||||
model="bedrock/anthropic.claude-instant-v1",
|
||||
model_id="provisioned-model-arn",
|
||||
messages=[{"content": "Hello, how are you?", "role": "user"}]
|
||||
)
|
||||
```
|
||||
|
||||
Embedding
|
||||
```python
|
||||
import litellm
|
||||
response = litellm.embedding(
|
||||
model="bedrock/amazon.titan-embed-text-v1",
|
||||
model_id="provisioned-model-arn",
|
||||
input=["hi"],
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
## Supported AWS Bedrock Models
|
||||
Here's an example of using a bedrock model with LiteLLM
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue