scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 115

boto3 testing + docs

Browse source
This commit is contained in:
ishaan-jaff 2023-09-04 12:14:24 -07:00
parent 38b0772194
commit 8f07c80733
2 changed files with 2 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
docs/my-website/docs/completion/supported.md
View file

@ -137,7 +137,7 @@ Example TogetherAI Usage - Note: liteLLM supports all models deployed on Togethe
### AWS Sagemaker Models
https://aws.amazon.com/sagemaker/ Use liteLLM to easily call custom LLMs on Sagemaker
### Requirements using Sagemaker with LiteLLM
#### Requirements using Sagemaker with LiteLLM
* `pip install boto3`
* Set the following AWS credentials as .env variables (Sagemaker auth: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html)
@ -149,6 +149,7 @@ https://aws.amazon.com/sagemaker/ Use liteLLM to easily call custom LLMs on Sage
| Model Name | Function Call | Required OS Variables |
|------------------|--------------------------------------------|------------------------------------|
| Llama2 7B | `completion(model='sagemaker/jumpstart-dft-meta-textgeneration-llama-2-7b, messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
| Custom LLM Endpoint | `completion(model='sagemaker/your-endpoint, messages=messages)` | `os.environ['AWS_ACCESS_KEY_ID']`, `os.environ['AWS_SECRET_ACCESS_KEY']`, `os.environ['AWS_REGION_NAME']` |
### Baseten Models
Baseten provides infrastructure to deploy and serve ML models https://www.baseten.co/. Use liteLLM to easily call models deployed on Baseten.

2
litellm/tests/test_completion.py
View file

@ -404,8 +404,6 @@ def test_completion_sagemaker():
print(response)
except Exception as e:
pytest.fail(f"Error occurred: {e}")
test_completion_sagemaker()
# def test_vertex_ai():
# model_name = "chat-bison"