scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-09 03:18:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 108

update docs

Browse source
This commit is contained in:
ishaan-jaff 2023-09-29 15:28:46 -07:00
parent a085812365
commit b8a35168ce
2 changed files with 14 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
docs/my-website/docs/providers/bedrock.md
View file

@ -17,21 +17,22 @@ os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = ""
response = completion(
model="bedrock/amazon.titan-tg1-large",
messages=[{ "content": "Hello, how are you?","role": "user"}],
temperature=0.2,
max_tokens=80,
model="bedrock/anthropic.claude-instant-v1",
messages=[{ "content": "Hello, how are you?","role": "user"}]
)
```
## Supported AWS Bedrock Models
Here's an example of using a bedrock model with LiteLLM
| Model Name | Function Call | Required OS Variables |
|------------------|--------------------------------------------|------------------------------------|
| Titan Text Large | `completion(model='bedrock/amazon.titan-tg1-large', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
| AI21 J2-Ultra | `completion(model='bedrock/ai21.j2-ultra', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
| AI21 J2-Mid | `completion(model='bedrock/ai21.j2-mid', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
| Model Name | Command | Environment Variables |
|--------------------------|------------------------------------------------------------------|---------------------------------------------------------------------|
| Anthropic Claude-V2 | `completion(model='bedrock/anthropic.claude-v2', messages=messages)` | `os.environ['ANTHROPIC_ACCESS_KEY_ID']`, `os.environ['ANTHROPIC_SECRET_ACCESS_KEY']` |
| Anthropic Claude-Instant V1 | `completion(model='bedrock/anthropic.claude-instant-v1', messages=messages)` | `os.environ['ANTHROPIC_ACCESS_KEY_ID']`, `os.environ['ANTHROPIC_SECRET_ACCESS_KEY']` |
| Anthropic Claude-V1 | `completion(model='bedrock/anthropic.claude-v1', messages=messages)` | `os.environ['ANTHROPIC_ACCESS_KEY_ID']`, `os.environ['ANTHROPIC_SECRET_ACCESS_KEY']` |
| AI21 J2-Ultra | `completion(model='bedrock/ai21.j2-ultra', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
| AI21 J2-Mid | `completion(model='bedrock/ai21.j2-mid', messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
## Streaming
Bedrock currently supports streaming for the following llms:
@ -47,10 +48,8 @@ os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = ""
response = completion(
model="bedrock/amazon.titan-tg1-large",
model="bedrock/anthropic.claude-instant-v1",
messages=[{ "content": "Hello, how are you?","role": "user"}],
temperature=0.2,
max_tokens=80,
stream=True
)

6
docs/my-website/sidebars.js
View file

@ -60,6 +60,9 @@ const sidebars = {
"providers/vertex",
"providers/palm",
"providers/anthropic",
"providers/aws_sagemaker",
"providers/bedrock",
"providers/ollama",
"providers/vllm",
"providers/deepinfra",
"providers/ai21",
@ -67,12 +70,9 @@ const sidebars = {
"providers/replicate",
"providers/cohere",
"providers/togetherai",
"providers/aws_sagemaker",
"providers/bedrock",
"providers/aleph_alpha",
"providers/baseten",
"providers/openrouter",
"providers/ollama",
"providers/custom",
"providers/custom_openai_proxy",
"providers/petals",