From 79e43088097066496b44f2121c1fb87d3980ab5d Mon Sep 17 00:00:00 2001
From: Timothy Lowrimore <tlowrimore@heroku.com>
Date: Fri, 25 Jul 2025 16:02:26 -0600
Subject: [PATCH] adds provider docs

---
 docs/my-website/docs/providers/heroku.md | 77 ++++++++++++++++++++++++
 1 file changed, 77 insertions(+)
 create mode 100644 docs/my-website/docs/providers/heroku.md

diff --git a/docs/my-website/docs/providers/heroku.md b/docs/my-website/docs/providers/heroku.md
new file mode 100644
index 00000000000..d10387f958a
--- /dev/null
+++ b/docs/my-website/docs/providers/heroku.md
@@ -0,0 +1,77 @@
+# Heroku
+
+## Provision a Model
+
+To use the Heroku provider for LiteLLM, you must first configure a Heroku app, and attach one of the models listed in the [Supported Models](#supported-models) section.
+
+To get configure a Heroku app with an attached model, please refer to [Heroku's documentation](https://devcenter.heroku.com/articles/heroku-inference).
+
+## Supported Models
+
+The Heroku provider for LiteLLM currently, only supports the following models for the [`v1/chat/completions`](https://devcenter.heroku.com/articles/heroku-inference-api-v1-chat-completions) endpoint:
+
+| Model                             | Region  |
+|-----------------------------------|---------|
+| [`heroku/claude-sonnet-4`](https://devcenter.heroku.com/articles/heroku-inference-api-model-claude-4-sonnet)          | US, EU  |
+| [`heroku/claude-3-7-sonnet`](https://devcenter.heroku.com/articles/heroku-inference-api-model-claude-3-7-sonnet)        | US, EU  |
+| [`heroku/claude-3-5-sonnet-latest`](https://devcenter.heroku.com/articles/heroku-inference-api-model-claude-3-5-sonnet-latest) | US      |
+| [`heroku/claude-3-5-haiku`](https://devcenter.heroku.com/articles/heroku-inference-api-model-claude-3-5-haiku)         | US      |
+| [`heroku/claude-3`](https://devcenter.heroku.com/articles/heroku-inference-api-model-claude-3-haiku)                 | EU      |
+
+## Environment Variables
+
+When a model is attached to a Heroku app, three config variables are set:
+
+- `INFERENCE_KEY`: The API key used for authenticating requests to the model.
+- `INFERENCE_MODEL_ID`: The name of the model. E.g. `claude-3-5-haiku`.
+- `INFERENCE_URL`: The base URL for calling the model.
+
+It is important to note that the values for `INFERENCE_KEY` and `INFERENCE_URL` will be required for making calls to your model. More details follow in the [Usage Examples](#usage-examples) section.
+
+For a deeper explanation of these variables, see the official [Heroku documentation](https://devcenter.heroku.com/articles/heroku-inference#model-resource-config-vars).
+
+## Usage Examples
+### Using Config Variables
+
+The Heroku provider is aware of the following config variables, and will use them, if present:
+
+- `HEROKU_API_KEY`: This value corresponds to the [`api_key` param](https://docs.litellm.ai/docs/set_keys#litellmapi_key). Set this to the value of Heroku's `INFERENCE_KEY` config variable.
+- `HEROKU_API_BASE`: This value corresponds to the [`api_base` param](https://docs.litellm.ai/docs/set_keys#litellmapi_base). Set this to the value of Heroku's `INFERENCE_URL` config variable.
+
+In this example, we don't explicitly pass the `api_key` and `api_base`. We, instead, set the config variables which will be used by the Heroku provider.
+
+```python
+import os
+from litellm import completion
+
+os.environ["HEROKU_API_BASE"] = "https://us.inference.heroku.com"
+os.environ["HEROKU_API_KEY"] = "fake-heroku-key"
+
+response = completion(
+    model="heroku/claude-3-5-haiku",
+    messages=[
+        {"role": "user", "content": "write code for saying hey from LiteLLM"}
+    ]
+)
+
+print(response)
+```
+
+### Explicitly Setting `api_key` and `api_base`
+
+```python
+from litellm import completion
+
+response = completion(
+    model="heroku/claude-sonnet-4",
+    api_key="fake-heroku-key",
+    api_base="https://us.inference.heroku.com",
+    messages=[
+        {"role": "user", "content": "write code for saying hey from LiteLLM"}
+    ],
+)
+```
+
+## Misc
+
+Note that in both of the above examples, the model name has the `heroku/` prefix. This is necessary, as it allows LiteLLM to know what model provider to use.
\ No newline at end of file