mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
feat(proxy): client-side provider API key precedence for Anthropic /v1/messages
- Add forward_llm_provider_auth_headers support from litellm_settings - When enabled, client x-api-key takes precedence over deployment keys - Forward x-api-key when x-litellm-api-key or Authorization used for auth - Fix duplicate patch lines in test_byok_oauth_endpoints.py - Add Claude Code BYOK documentation with /login and ANTHROPIC_CUSTOM_HEADERS - Add unit tests for clean_headers x-api-key forwarding logic - Sync model_prices backup (pre-commit hook) Made-with: Cursor
This commit is contained in:
parent
118cad8b54
commit
159c477c18
7 changed files with 292 additions and 2 deletions
|
|
@ -112,6 +112,8 @@ general_settings:
|
|||
forward_llm_provider_auth_headers: true # Enable BYOK
|
||||
```
|
||||
|
||||
For **Claude Code** with `/login` and your own Anthropic key, see [Claude Code BYOK](../tutorials/claude_code_byok.md). Use `ANTHROPIC_CUSTOM_HEADERS="x-litellm-api-key: sk-12345"` to pass your LiteLLM key while your Anthropic key (from `/login`) is forwarded as `x-api-key`.
|
||||
|
||||
Client request:
|
||||
```bash
|
||||
curl -X POST "http://localhost:4000/v1/messages" \
|
||||
|
|
|
|||
123
docs/my-website/docs/tutorials/claude_code_byok.md
Normal file
123
docs/my-website/docs/tutorials/claude_code_byok.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# Claude Code with Bring Your Own Key (BYOK)
|
||||
|
||||
Use Claude Code with your own Anthropic API key through the LiteLLM proxy. When you use Claude's `/login` with your Anthropic account, your API key is sent as `x-api-key`. With BYOK enabled, LiteLLM forwards your key to Anthropic instead of using proxy-configured keys — so you pay Anthropic directly while still benefiting from LiteLLM's routing, logging, and guardrails.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Claude Code `/login`** — You sign in with your Anthropic account; Claude Code sends your Anthropic API key as `x-api-key`.
|
||||
2. **LiteLLM authentication** — You pass your LiteLLM proxy key via `ANTHROPIC_CUSTOM_HEADERS` so the proxy can authenticate and track your usage.
|
||||
3. **Key forwarding** — With `forward_llm_provider_auth_headers: true`, LiteLLM forwards your `x-api-key` to Anthropic, giving it precedence over any proxy-configured keys.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) installed
|
||||
- Anthropic API key (from [console.anthropic.com](https://console.anthropic.com))
|
||||
- LiteLLM proxy with a virtual key for authentication
|
||||
|
||||
## Step 1: Configure LiteLLM Proxy
|
||||
|
||||
Enable forwarding of LLM provider auth headers so your Anthropic key takes precedence:
|
||||
|
||||
```yaml title="config.yaml"
|
||||
model_list:
|
||||
- model_name: claude-sonnet-4-5
|
||||
litellm_params:
|
||||
model: anthropic/claude-sonnet-4-5
|
||||
# No api_key needed — client's key will be used
|
||||
|
||||
litellm_settings:
|
||||
forward_llm_provider_auth_headers: true # Required for BYOK
|
||||
```
|
||||
|
||||
:::info Why `forward_llm_provider_auth_headers`?
|
||||
|
||||
By default, LiteLLM strips `x-api-key` from client requests for security. Setting this to `true` allows client-provided provider keys (like your Anthropic key from `/login`) to be forwarded to Anthropic, overriding any proxy-configured keys.
|
||||
|
||||
:::
|
||||
|
||||
## Step 2: Create a LiteLLM Virtual Key
|
||||
|
||||
Create a virtual key in the LiteLLM UI or via API.
|
||||
```bash
|
||||
# Example: Create key via API
|
||||
curl -X POST "http://localhost:4000/key/generate" \
|
||||
-H "Authorization: Bearer sk-your-master-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"key_alias": "claude-code-byok", "models": ["claude-sonnet-4-5"]}'
|
||||
```
|
||||
|
||||
## Step 3: Configure Claude Code
|
||||
|
||||
Set environment variables so Claude Code uses LiteLLM and sends your LiteLLM key for proxy auth:
|
||||
|
||||
```bash
|
||||
# Point Claude Code to your LiteLLM proxy
|
||||
export ANTHROPIC_BASE_URL="http://localhost:4000"
|
||||
|
||||
# Model name from your config
|
||||
export ANTHROPIC_MODEL="claude-sonnet-4-5"
|
||||
|
||||
# LiteLLM proxy auth — this is added to every request
|
||||
# Use x-litellm-api-key so the proxy authenticates you; your Anthropic key goes via x-api-key from /login
|
||||
export ANTHROPIC_CUSTOM_HEADERS="x-litellm-api-key: sk-12345"
|
||||
```
|
||||
|
||||
Replace `sk-12345` with your actual LiteLLM virtual key.
|
||||
|
||||
:::tip Multiple headers
|
||||
|
||||
For multiple headers, use newline-separated values:
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_CUSTOM_HEADERS="x-litellm-api-key: sk-12345
|
||||
x-litellm-user-id: my-user-id"
|
||||
```
|
||||
|
||||
:::
|
||||
|
||||
## Step 4: Sign In with Claude Code
|
||||
|
||||
1. Launch Claude Code:
|
||||
|
||||
```bash
|
||||
claude
|
||||
```
|
||||
|
||||
2. Use **`/login`** and sign in with your Anthropic account (or use your API key directly).
|
||||
|
||||
3. Claude Code will send:
|
||||
- `x-api-key`: Your Anthropic API key (from `/login`)
|
||||
- `x-litellm-api-key`: Your LiteLLM key (from `ANTHROPIC_CUSTOM_HEADERS`)
|
||||
|
||||
4. LiteLLM authenticates you via `x-litellm-api-key`, then forwards `x-api-key` to Anthropic. Your Anthropic key takes precedence over any proxy-configured key.
|
||||
|
||||
## Summary
|
||||
|
||||
| Header | Source | Purpose |
|
||||
|--------|--------|---------|
|
||||
| `x-api-key` | Claude Code `/login` (Anthropic key) | Sent to Anthropic for API calls |
|
||||
| `x-litellm-api-key` | `ANTHROPIC_CUSTOM_HEADERS` | Proxy authentication, tracking, rate limits |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Requests fail with "invalid x-api-key"
|
||||
|
||||
- Ensure `forward_llm_provider_auth_headers: true` is set in `litellm_settings` (or `general_settings`).
|
||||
- Restart the LiteLLM proxy after changing the config.
|
||||
- Verify you completed `/login` in Claude Code so your Anthropic key is being sent.
|
||||
|
||||
### Proxy returns 401
|
||||
|
||||
- Check that `ANTHROPIC_CUSTOM_HEADERS` includes `x-litellm-api-key: <your-key>`.
|
||||
- Ensure the LiteLLM key is valid and has access to the model.
|
||||
|
||||
### Proxy key is used instead of my Anthropic key
|
||||
|
||||
- Confirm `forward_llm_provider_auth_headers: true` is in your config.
|
||||
- The setting can be in `litellm_settings` or `general_settings` depending on your config structure.
|
||||
- Enable debug logging: `LITELLM_LOG=DEBUG` to see which key is being forwarded.
|
||||
|
||||
## Related
|
||||
|
||||
- [Forward Client Headers](./../proxy/forward_client_headers.md) — Full BYOK and header forwarding docs
|
||||
- [Claude Code Max Subscription](./claude_code_max_subscription.md) — Using Claude Code with OAuth/Max subscription through LiteLLM
|
||||
BIN
docs/my-website/img/claude_code_byok_screenshot.png
Normal file
BIN
docs/my-website/img/claude_code_byok_screenshot.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 99 KiB |
|
|
@ -154,6 +154,7 @@ const sidebars = {
|
|||
items: [
|
||||
"tutorials/claude_responses_api",
|
||||
"tutorials/claude_code_max_subscription",
|
||||
"tutorials/claude_code_byok",
|
||||
"tutorials/claude_code_customer_tracking",
|
||||
"tutorials/claude_code_prompt_cache_routing",
|
||||
"tutorials/claude_code_websearch",
|
||||
|
|
|
|||
|
|
@ -258,9 +258,20 @@ def clean_headers(
|
|||
headers: Headers,
|
||||
litellm_key_header_name: Optional[str] = None,
|
||||
forward_llm_provider_auth_headers: bool = False,
|
||||
authenticated_with_header: Optional[str] = None,
|
||||
) -> dict:
|
||||
"""
|
||||
Removes litellm api key from headers
|
||||
|
||||
Args:
|
||||
headers: Request headers
|
||||
litellm_key_header_name: Custom header name for LiteLLM API key
|
||||
forward_llm_provider_auth_headers: Whether to forward provider auth headers
|
||||
authenticated_with_header: Which header was used for LiteLLM authentication
|
||||
(e.g., "x-litellm-api-key", "authorization", "x-api-key")
|
||||
|
||||
Returns:
|
||||
Cleaned headers dict
|
||||
"""
|
||||
from litellm.llms.anthropic.common_utils import is_anthropic_oauth_key
|
||||
|
||||
|
|
@ -273,6 +284,13 @@ def clean_headers(
|
|||
|
||||
if header_lower == "authorization" and is_anthropic_oauth_key(value):
|
||||
clean_headers[header] = value
|
||||
# Special handling for x-api-key: forward it based on authenticated_with_header
|
||||
elif header_lower == "x-api-key":
|
||||
if (
|
||||
forward_llm_provider_auth_headers
|
||||
and (authenticated_with_header is None or authenticated_with_header.lower() != "x-api-key")
|
||||
):
|
||||
clean_headers[header] = value
|
||||
elif (
|
||||
forward_llm_provider_auth_headers and header_lower in _SPECIAL_HEADERS_CACHE
|
||||
):
|
||||
|
|
@ -280,6 +298,9 @@ def clean_headers(
|
|||
continue
|
||||
if header_lower == "authorization":
|
||||
continue
|
||||
# Never forward x-litellm-api-key (it's for proxy auth only)
|
||||
if header_lower == "x-litellm-api-key":
|
||||
continue
|
||||
clean_headers[header] = value
|
||||
# Check if header should be excluded: either in special headers cache or matches custom litellm key
|
||||
elif header_lower not in _SPECIAL_HEADERS_CACHE and (
|
||||
|
|
@ -868,6 +889,20 @@ async def add_litellm_data_to_request( # noqa: PLR0915
|
|||
forward_llm_auth = general_settings.get(
|
||||
"forward_llm_provider_auth_headers", False
|
||||
)
|
||||
if not forward_llm_auth:
|
||||
forward_llm_auth = getattr(litellm, "forward_llm_provider_auth_headers", False)
|
||||
# Determine which header was used for authentication
|
||||
# This enables forwarding provider keys (e.g., x-api-key) when they weren't used for LiteLLM auth
|
||||
authenticated_with_header = None
|
||||
if "x-litellm-api-key" in request.headers:
|
||||
# If x-litellm-api-key is present, it was used for auth
|
||||
authenticated_with_header = "x-litellm-api-key"
|
||||
elif "authorization" in request.headers:
|
||||
# Authorization header was used for auth
|
||||
authenticated_with_header = "authorization"
|
||||
else:
|
||||
# x-api-key or another header was used for auth
|
||||
authenticated_with_header = "x-api-key"
|
||||
|
||||
_headers: Dict[str, str] = clean_headers(
|
||||
request.headers,
|
||||
|
|
@ -877,10 +912,18 @@ async def add_litellm_data_to_request( # noqa: PLR0915
|
|||
else None
|
||||
),
|
||||
forward_llm_provider_auth_headers=forward_llm_auth,
|
||||
authenticated_with_header=authenticated_with_header,
|
||||
)
|
||||
verbose_proxy_logger.debug(f"Request Headers: {_headers}")
|
||||
verbose_proxy_logger.debug(f"Raw Headers: {_raw_headers}")
|
||||
|
||||
|
||||
if forward_llm_auth and "x-api-key" in _headers:
|
||||
data["api_key"] = _headers["x-api-key"]
|
||||
verbose_proxy_logger.debug(
|
||||
"Setting client-provided x-api-key as api_key parameter (will override deployment key)"
|
||||
)
|
||||
|
||||
##########################################################
|
||||
# Init - Proxy Server Request
|
||||
# we do this as soon as entering so we track the original request
|
||||
|
|
|
|||
|
|
@ -503,3 +503,126 @@ class TestProxyOAuthHeaderForwarding:
|
|||
psh = data["provider_specific_header"]
|
||||
assert psh["extra_headers"]["authorization"] == f"Bearer {FAKE_OAUTH_TOKEN}"
|
||||
assert psh["extra_headers"]["anthropic-beta"] == "oauth-2025-04-20"
|
||||
|
||||
def test_clean_headers_forwards_x_api_key_when_authenticated_with_litellm_key(self):
|
||||
"""clean_headers should forward x-api-key when user authenticated with x-litellm-api-key and forward_llm_provider_auth_headers=True."""
|
||||
from starlette.datastructures import Headers
|
||||
|
||||
from litellm.proxy.litellm_pre_call_utils import clean_headers
|
||||
|
||||
raw_headers = Headers(
|
||||
raw=[
|
||||
(b"x-litellm-api-key", b"sk-litellm-proxy-key"),
|
||||
(b"x-api-key", b"sk-ant-api03-client-key"),
|
||||
(b"content-type", b"application/json"),
|
||||
]
|
||||
)
|
||||
cleaned = clean_headers(
|
||||
raw_headers,
|
||||
forward_llm_provider_auth_headers=True,
|
||||
authenticated_with_header="x-litellm-api-key",
|
||||
)
|
||||
|
||||
# x-api-key should be forwarded (it's a provider key, not used for auth)
|
||||
assert "x-api-key" in cleaned
|
||||
assert cleaned["x-api-key"] == "sk-ant-api03-client-key"
|
||||
# x-litellm-api-key should be excluded (special header)
|
||||
assert "x-litellm-api-key" not in cleaned
|
||||
assert cleaned["content-type"] == "application/json"
|
||||
|
||||
def test_clean_headers_excludes_x_api_key_when_used_for_auth(self):
|
||||
"""clean_headers should exclude x-api-key when it was used for LiteLLM authentication."""
|
||||
from starlette.datastructures import Headers
|
||||
|
||||
from litellm.proxy.litellm_pre_call_utils import clean_headers
|
||||
|
||||
raw_headers = Headers(
|
||||
raw=[
|
||||
(b"x-api-key", b"sk-litellm-proxy-key"),
|
||||
(b"content-type", b"application/json"),
|
||||
]
|
||||
)
|
||||
cleaned = clean_headers(raw_headers, authenticated_with_header="x-api-key")
|
||||
|
||||
# x-api-key should be excluded (was used for LiteLLM auth)
|
||||
assert "x-api-key" not in cleaned
|
||||
assert cleaned["content-type"] == "application/json"
|
||||
|
||||
def test_clean_headers_forwards_x_api_key_when_authenticated_with_authorization(
|
||||
self,
|
||||
):
|
||||
"""clean_headers should forward x-api-key when user authenticated with Authorization header and forward_llm_provider_auth_headers=True."""
|
||||
from starlette.datastructures import Headers
|
||||
|
||||
from litellm.proxy.litellm_pre_call_utils import clean_headers
|
||||
|
||||
raw_headers = Headers(
|
||||
raw=[
|
||||
(b"authorization", b"Bearer sk-litellm-proxy-key"),
|
||||
(b"x-api-key", b"sk-ant-api03-client-key"),
|
||||
(b"content-type", b"application/json"),
|
||||
]
|
||||
)
|
||||
cleaned = clean_headers(
|
||||
raw_headers,
|
||||
forward_llm_provider_auth_headers=True,
|
||||
authenticated_with_header="authorization",
|
||||
)
|
||||
|
||||
# x-api-key should be forwarded (it's a provider key, not used for auth)
|
||||
assert "x-api-key" in cleaned
|
||||
assert cleaned["x-api-key"] == "sk-ant-api03-client-key"
|
||||
# authorization should be excluded (was used for auth, not OAuth)
|
||||
assert "authorization" not in cleaned
|
||||
assert cleaned["content-type"] == "application/json"
|
||||
|
||||
def test_clean_headers_x_api_key_without_authenticated_header_param(self):
|
||||
"""clean_headers should exclude x-api-key when authenticated_with_header is None."""
|
||||
from starlette.datastructures import Headers
|
||||
|
||||
from litellm.proxy.litellm_pre_call_utils import clean_headers
|
||||
|
||||
raw_headers = Headers(
|
||||
raw=[
|
||||
(b"x-api-key", b"sk-ant-api03-key"),
|
||||
(b"content-type", b"application/json"),
|
||||
]
|
||||
)
|
||||
cleaned = clean_headers(raw_headers, authenticated_with_header=None)
|
||||
|
||||
# x-api-key should be excluded (no authenticated_with_header means we can't determine)
|
||||
assert "x-api-key" not in cleaned
|
||||
assert cleaned["content-type"] == "application/json"
|
||||
|
||||
def test_clean_headers_forwards_x_api_key_with_forward_flag_and_litellm_auth(
|
||||
self,
|
||||
):
|
||||
"""clean_headers should forward x-api-key when both forward_llm_provider_auth_headers=True
|
||||
and authenticated_with_header indicates different header was used for auth."""
|
||||
from starlette.datastructures import Headers
|
||||
|
||||
from litellm.proxy.litellm_pre_call_utils import clean_headers
|
||||
|
||||
raw_headers = Headers(
|
||||
raw=[
|
||||
(b"x-litellm-api-key", b"sk-litellm-proxy-key"),
|
||||
(b"x-api-key", b"sk-ant-api03-client-key"),
|
||||
(b"x-goog-api-key", b"google-key-123"),
|
||||
(b"content-type", b"application/json"),
|
||||
]
|
||||
)
|
||||
cleaned = clean_headers(
|
||||
raw_headers,
|
||||
forward_llm_provider_auth_headers=True,
|
||||
authenticated_with_header="x-litellm-api-key",
|
||||
)
|
||||
|
||||
# x-api-key should be forwarded (provider key, not used for auth)
|
||||
assert "x-api-key" in cleaned
|
||||
assert cleaned["x-api-key"] == "sk-ant-api03-client-key"
|
||||
# x-goog-api-key should also be forwarded (forward flag is True)
|
||||
assert "x-goog-api-key" in cleaned
|
||||
assert cleaned["x-goog-api-key"] == "google-key-123"
|
||||
# x-litellm-api-key should be excluded (special header)
|
||||
assert "x-litellm-api-key" not in cleaned
|
||||
assert cleaned["content-type"] == "application/json"
|
||||
|
|
|
|||
|
|
@ -477,7 +477,6 @@ async def test_check_byok_credential_missing_credential():
|
|||
with patch(
|
||||
"litellm.proxy._experimental.mcp_server.db.get_user_credential",
|
||||
new=AsyncMock(return_value=None),
|
||||
), patch("litellm.proxy.proxy_server.prisma_client", mock_prisma):
|
||||
), patch("litellm.proxy.proxy_server.prisma_client", mock_prisma):
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
await _check_byok_credential(server, user_auth)
|
||||
|
|
@ -511,7 +510,6 @@ async def test_check_byok_credential_has_credential():
|
|||
with patch(
|
||||
"litellm.proxy._experimental.mcp_server.db.get_user_credential",
|
||||
new=AsyncMock(return_value="some-credential-value"),
|
||||
), patch("litellm.proxy.proxy_server.prisma_client", mock_prisma):
|
||||
), patch("litellm.proxy.proxy_server.prisma_client", mock_prisma):
|
||||
# Should not raise
|
||||
await _check_byok_credential(server, user_auth)
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue