Enhance Bedrock Provider Configuration and Header Management

- Added `forward_client_headers_to_llm_api` setting in the Bedrock documentation to facilitate client-side header forwarding.
- Updated `completion` function to use merged headers instead of original `extra_headers`.
- Improved request handling in `BedrockConverseLLM` and `AmazonInvokeConfig` to ensure proper header management for `anthropic-beta` parameters.
- Refactored request transformation logic to return the transformed request for better clarity and functionality.
This commit is contained in:
Jugal Bhatt 2025-08-13 13:41:34 -07:00
parent 26e62c9bd8
commit 4201f0aa79
4 changed files with 13 additions and 2 deletions

View file

@ -687,6 +687,9 @@ model_list:
model: bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0
extra_headers:
anthropic-beta: "computer-use-2024-10-22,context-1m-2025-08-07"
general_settings:
forward_client_headers_to_llm_api: true # 👈 Required for client-side header forwarding
```
**Set on Request**
@ -711,6 +714,10 @@ response = client.chat.completions.create(
)
```
:::info
**For client-side header forwarding**: When using the proxy and sending `anthropic-beta` headers from the client (like the OpenAI SDK), you need to enable `forward_client_headers_to_llm_api: true` in your proxy's `general_settings`. This tells the proxy to extract headers from HTTP requests and forward them to the underlying LLM provider.
:::
</TabItem>
</Tabs>

View file

@ -189,6 +189,7 @@ class BedrockConverseLLM(BaseAWSLLM):
headers=headers,
)
data = json.dumps(request_data)
prepped = self.get_request_headers(
credentials=credentials,
aws_region_name=litellm_params.get("aws_region_name") or "us-west-2",
@ -395,6 +396,7 @@ class BedrockConverseLLM(BaseAWSLLM):
headers=extra_headers,
)
data = json.dumps(_data)
prepped = self.get_request_headers(
credentials=credentials,
aws_region_name=aws_region_name,

View file

@ -190,13 +190,15 @@ class AmazonInvokeConfig(BaseConfig, BaseAWSLLM):
] = True # cohere requires stream = True in inference params
request_data = {"prompt": prompt, **inference_params}
elif provider == "anthropic":
return litellm.AmazonAnthropicClaudeConfig().transform_request(
transformed_request = litellm.AmazonAnthropicClaudeConfig().transform_request(
model=model,
messages=messages,
optional_params=optional_params,
litellm_params=litellm_params,
headers=headers,
)
return transformed_request
elif provider == "nova":
return litellm.AmazonInvokeNovaConfig().transform_request(
model=model,

View file

@ -2981,7 +2981,7 @@ def completion( # type: ignore # noqa: PLR0915
logger_fn=logger_fn,
encoding=encoding,
logging_obj=logging,
extra_headers=extra_headers,
extra_headers=headers, # Use merged headers instead of original extra_headers
timeout=timeout,
acompletion=acompletion,
client=client,