fix(bedrock): keep x-amzn-RequestId on chat error responses (#40089)

* fix(bedrock): keep x-amzn-RequestId on chat error responses

Bedrock chat error paths built BedrockError from only a status code and a
message, so the provider response headers were gone before exception mapping
ran and the proxy had nothing to forward. AWS support needs x-amzn-RequestId
to investigate a server-side error.

- converse and invoke chat handlers pass the real headers and response when
  they turn an httpx.HTTPStatusError into a BedrockError, and read the body
  through error_response_text so a streamed body nobody read does not throw
- every bedrock chat get_error_class honors the headers it is already handed:
  invoke, moonshot, bedrock-hosted openai, agentcore and the invoke agent
- BedrockError carries those headers into the response it synthesizes when a
  caller has headers but no response, skipping values httpx cannot carry
- the bedrock 500 mapping forwards the provider response like its 4xx and 503
  siblings instead of fabricating a blank one

The proxy now returns llm_provider-x-amzn-requestid on Bedrock chat errors.

* fix(bedrock): keep request-id on text-classified errors

The context-window and image branches of _map_bedrock_exception built their
litellm exception without the provider response, so a Bedrock 400 classified
by its body text lost x-amzn-RequestId while the sibling branches kept it.

Also narrows the new BedrockError types and trims its docstrings.

* chore(bedrock): drop the docstrings on the new error helpers

* fix(bedrock): keep request-id on every error path that has one

The ticket's root cause is that every BedrockError raise site under
litellm/llms/bedrock/ was built from status and message alone. The first
commits covered the chat and invoke handlers; this covers the rest.

Embeddings, rerank, image generation, image edit, count tokens, search and
the transformation layers now hand on the provider response or its headers,
and both bedrock_mantle configs return a BedrockError instead of the
OpenAI error that drops them.

Two blockers surfaced while verifying the streaming path. The trailing
`except Exception` in make_call and make_sync_call swallowed the BedrockError
raised a few lines above, relabelling a provider status as a 500, and the
non-200 branch read an unread streamed body, which throws.

The raise sites left alone have no provider response to carry: timeouts,
credential and config errors, and mid-stream event frames.

* fix(bedrock): forward provider headers from the count tokens route

The count tokens route converts BedrockError into an HTTPException, and dropped
the headers the handler had just kept, so that route still lost the request id.

get_response_headers now takes a Mapping so an httpx.Headers can be handed to it
without a copy.

* fix(bedrock): classify every bedrock surface through BedrockError

Eleven bedrock configs still inherited a provider-agnostic get_error_class
that builds a blank response, so the request id was gone before the proxy
read it. Claude platform, bedrock anthropic-messages, both image edit
configs, passthrough, realtime, vector stores and agentcore search now
return BedrockError, and a parametrized audit drives all 36 configs.

* fix(proxy): keep provider headers on the httpx status error branch

_handle_llm_api_exception forwards safe_headers on every branch except the
httpx.HTTPStatusError one, which the bedrock passthrough route reaches, so
the request id was dropped before the client saw the response.

* fix(bedrock): keep the request id on the timeout mappings

Timeout takes no response argument, so the three bedrock timeout branches
dropped the provider headers even when the upstream answered 408 or 504
with an x-amzn-RequestId. They now ride on the exception, already
llm_provider-prefixed, which is the form the proxy emits.

* fix(bedrock): keep the provider response on mapped timeouts

The previous round attached llm_provider-prefixed headers directly to the
Timeout. That shadowed the raw upstream headers for _get_response_headers,
so router cooldown and fallback cooldown stopped honouring retry-after on
bedrock 408/504 replies.

Give Timeout an optional response instead, the way every other mapped
bedrock exception already carries one. Retry logic reads the raw
retry-after off the response, and the proxy prefixes those headers on the
way out, so clients still see llm_provider-x-amzn-requestid.

* chore(bedrock): drop the explanatory comment on Timeout.response
This commit is contained in:
yucheng-berri 2026-09-07 17:16:47 -07:00 committed by GitHub
parent 9d0c9b9382
commit 1009976c49
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
39 changed files with 1100 additions and 35 deletions

View file

@ -338,6 +338,7 @@ class Timeout(openai.APITimeoutError):
num_retries: int | None = None,
headers: dict | None = None,
exception_status_code: int | None = None,
response: httpx.Response | None = None,
):
request: Final = httpx.Request(
method="POST",
@ -352,6 +353,8 @@ class Timeout(openai.APITimeoutError):
self.max_retries = max_retries
self.num_retries = num_retries
self.headers = headers
if response is not None:
self.response = response
# custom function to convert to str
def __str__(self):

View file

@ -860,6 +860,7 @@ def _map_bedrock_exception(
message=mantle_context_window_message,
model=model,
llm_provider=custom_llm_provider,
response=getattr(original_exception, "response", None),
)
if (
"too many tokens" in error_str
@ -873,6 +874,7 @@ def _map_bedrock_exception(
message=f"BedrockException: Context Window Error - {error_str}",
model=model,
llm_provider="bedrock",
response=getattr(original_exception, "response", None),
)
elif "Conversation blocks and tool result blocks cannot be provided in the same turn." in error_str:
raise BadRequestError(
@ -924,12 +926,14 @@ def _map_bedrock_exception(
message=f"BedrockException: Timeout Error - {error_str}",
model=model,
llm_provider="bedrock",
response=getattr(original_exception, "response", None),
)
elif "Could not process image" in error_str:
raise litellm.InternalServerError(
message=f"BedrockException - {error_str}",
model=model,
llm_provider="bedrock",
response=getattr(original_exception, "response", None),
)
elif hasattr(original_exception, "status_code"):
if original_exception.status_code == 500:
@ -937,10 +941,7 @@ def _map_bedrock_exception(
message=f"BedrockException - {original_exception.message}",
llm_provider="bedrock",
model=model,
response=httpx.Response(
status_code=500,
request=httpx.Request(method="POST", url="https://api.openai.com/v1/"),
),
response=getattr(original_exception, "response", None),
)
elif original_exception.status_code == 401:
raise AuthenticationError(
@ -969,6 +970,7 @@ def _map_bedrock_exception(
model=model,
llm_provider=custom_llm_provider,
litellm_debug_info=extra_information,
response=getattr(original_exception, "response", None),
)
elif original_exception.status_code == 422:
raise BadRequestError(
@ -1001,6 +1003,7 @@ def _map_bedrock_exception(
llm_provider=custom_llm_provider,
litellm_debug_info=extra_information,
exception_status_code=original_exception.status_code,
response=getattr(original_exception, "response", None),
)

View file

@ -1,7 +1,8 @@
from collections.abc import Mapping
from typing import Final
def get_response_headers(_response_headers: dict | None = None) -> dict:
def get_response_headers(_response_headers: Mapping[str, str] | None = None) -> dict:
"""
Sets the Appropriate OpenAI headers for the response and forward all headers as llm_provider-{header}
@ -31,7 +32,7 @@ def get_response_headers(_response_headers: dict | None = None) -> dict:
return {**llm_provider_headers, **openai_headers}
def _get_llm_provider_headers(response_headers: dict) -> dict:
def _get_llm_provider_headers(response_headers: Mapping[str, str]) -> dict:
"""
Adds a llm_provider-{header} to all headers that are not already prefixed with llm_provider

View file

@ -667,7 +667,12 @@ class AmazonAgentCoreConfig(BaseConfig, BaseAWSLLM):
)
if response.status_code != 200:
raise BedrockError(status_code=response.status_code, message=str(response.read()))
raise BedrockError(
status_code=response.status_code,
message=str(response.read()),
headers=response.headers,
response=response,
)
# LOGGING
logging_obj.post_call(
@ -690,6 +695,7 @@ class AmazonAgentCoreConfig(BaseConfig, BaseAWSLLM):
raise BedrockError(
status_code=response.status_code,
message=f"AgentCore: Failed to read/parse JSON response body: {e}",
headers=response.headers,
)
parsed: Final = self._parse_json_response(response_json)
@ -880,7 +886,12 @@ class AmazonAgentCoreConfig(BaseConfig, BaseAWSLLM):
)
if response.status_code != 200:
raise BedrockError(status_code=response.status_code, message=str(await response.aread()))
raise BedrockError(
status_code=response.status_code,
message=str(await response.aread()),
headers=response.headers,
response=response,
)
# LOGGING
logging_obj.post_call(
@ -903,6 +914,7 @@ class AmazonAgentCoreConfig(BaseConfig, BaseAWSLLM):
raise BedrockError(
status_code=response.status_code,
message=f"AgentCore: Failed to read/parse JSON response body: {e}",
headers=response.headers,
)
parsed: Final = self._parse_json_response(response_json)
@ -1031,6 +1043,7 @@ class AmazonAgentCoreConfig(BaseConfig, BaseAWSLLM):
raise BedrockError(
message=f"Error processing response: {e}",
status_code=raw_response.status_code,
headers=raw_response.headers,
)
def validate_environment(
@ -1046,7 +1059,7 @@ class AmazonAgentCoreConfig(BaseConfig, BaseAWSLLM):
return headers
def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:
return BedrockError(status_code=status_code, message=error_message)
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def should_fake_stream(
self,

View file

@ -22,7 +22,7 @@ from litellm.types.utils import ModelResponse
from litellm.utils import CustomStreamWrapper
from ..base_aws_llm import BaseAWSLLM, Credentials, bedrock_bearer_token
from ..common_utils import BedrockError, _get_all_bedrock_regions
from ..common_utils import BedrockError, _get_all_bedrock_regions, error_response_text
from .invoke_handler import AWSEventStreamDecoder, MockResponseIterator, make_call
@ -66,7 +66,12 @@ def make_sync_call(
)
if response.status_code != 200:
raise BedrockError(status_code=response.status_code, message=str(response.read()))
raise BedrockError(
status_code=response.status_code,
message=str(response.read()),
headers=response.headers,
response=response,
)
if fake_stream:
model_response: Final[ModelResponse] = litellm.AmazonConverseConfig()._transform_response(
@ -247,7 +252,12 @@ class BedrockConverseLLM(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=error_response_text(err.response),
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")
@ -594,7 +604,12 @@ class BedrockConverseLLM(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=error_response_text(err.response),
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")

View file

@ -2255,6 +2255,7 @@ class AmazonConverseConfig(BaseConfig):
raise BedrockError(
message=f"Error converting to valid response block={e}. File an issue if litellm error - https://github.com/BerriAI/litellm/issues",
status_code=422,
headers=response.headers,
)
"""

View file

@ -470,6 +470,7 @@ class AmazonInvokeAgentConfig(BaseConfig, BaseAWSLLM):
raise BedrockError(
message=f"Error processing response: {e}",
status_code=raw_response.status_code,
headers=raw_response.headers,
)
def validate_environment(
@ -485,7 +486,7 @@ class AmazonInvokeAgentConfig(BaseConfig, BaseAWSLLM):
return headers
def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:
return BedrockError(status_code=status_code, message=error_message)
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def should_fake_stream(
self,

View file

@ -42,6 +42,7 @@ from litellm.types.utils import GenericStreamingChunk as GChunk
from ..common_utils import (
BedrockError,
build_bedrock_stream_error,
error_response_text,
get_bedrock_response_stream_shape,
get_bedrock_tool_name,
)
@ -184,7 +185,12 @@ async def make_call(
)
if response.status_code != 200:
raise BedrockError(status_code=response.status_code, message=response.text)
raise BedrockError(
status_code=response.status_code,
message=error_response_text(response),
headers=response.headers,
response=response,
)
if fake_stream:
model_response: Final[ModelResponse] = litellm.AmazonConverseConfig()._transform_response(
@ -228,9 +234,16 @@ async def make_call(
)
return completion_stream, response.headers
except BedrockError:
raise
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=error_response_text(err.response),
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")
except Exception as e:
@ -270,7 +283,12 @@ def make_sync_call(
)
if response.status_code != 200:
raise BedrockError(status_code=response.status_code, message=response.text)
raise BedrockError(
status_code=response.status_code,
message=error_response_text(response),
headers=response.headers,
response=response,
)
if fake_stream:
model_response: Final[ModelResponse] = litellm.AmazonConverseConfig()._transform_response(
@ -314,9 +332,16 @@ def make_sync_call(
)
return completion_stream, response.headers
except BedrockError:
raise
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=error_response_text(err.response),
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")
except Exception as e:

View file

@ -247,4 +247,4 @@ class AmazonMoonshotConfig(AmazonInvokeConfig, MoonshotChatConfig):
def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BedrockError:
"""Return the appropriate error class for Bedrock."""
return BedrockError(status_code=status_code, message=error_message)
return BedrockError(status_code=status_code, message=error_message, headers=headers)

View file

@ -182,4 +182,4 @@ class AmazonBedrockOpenAIConfig(OpenAIGPTConfig, BaseAWSLLM):
def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BedrockError:
"""Return the appropriate error class for Bedrock."""
return BedrockError(status_code=status_code, message=error_message)
return BedrockError(status_code=status_code, message=error_message, headers=headers)

View file

@ -212,6 +212,7 @@ class AmazonTwelveLabsPegasusConfig(AmazonInvokeConfig, BaseConfig):
raise BedrockError(
message=f"Error parsing response: {raw_response.text}, error: {e}",
status_code=raw_response.status_code,
headers=raw_response.headers,
)
verbose_logger.debug(
@ -241,6 +242,7 @@ class AmazonTwelveLabsPegasusConfig(AmazonInvokeConfig, BaseConfig):
raise BedrockError(
message=f"Error setting response content: {e}. Response: {completion_response}",
status_code=raw_response.status_code,
headers=raw_response.headers,
)
# Calculate usage from headers

View file

@ -295,7 +295,11 @@ class AmazonInvokeConfig(BaseConfig, BaseAWSLLM):
try:
completion_response: Final = raw_response.json()
except Exception:
raise BedrockError(message=raw_response.text, status_code=raw_response.status_code)
raise BedrockError(
message=raw_response.text,
status_code=raw_response.status_code,
headers=raw_response.headers,
)
verbose_logger.debug(
"bedrock invoke response % s",
json.dumps(completion_response, indent=4, default=str),
@ -363,6 +367,7 @@ class AmazonInvokeConfig(BaseConfig, BaseAWSLLM):
raise BedrockError(
message=f"Error processing={raw_response.text}, Received error={e}",
status_code=422,
headers=raw_response.headers,
)
try:
@ -384,6 +389,7 @@ class AmazonInvokeConfig(BaseConfig, BaseAWSLLM):
raise BedrockError(
message=f"Error parsing received text={outputText}.\nError-{e}",
status_code=raw_response.status_code,
headers=raw_response.headers,
)
## CALCULATING USAGE - bedrock returns usage in the headers
@ -431,7 +437,7 @@ class AmazonInvokeConfig(BaseConfig, BaseAWSLLM):
return merge_bedrock_invoke_headers(headers, guardrail_headers, metadata_headers, owned_names)
def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:
return BedrockError(status_code=status_code, message=error_message)
return BedrockError(status_code=status_code, message=error_message, headers=headers)
@track_llm_api_timing()
async def get_async_custom_stream_wrapper(

View file

@ -1,7 +1,10 @@
from typing import Final
import httpx
import litellm
from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.secret_managers.main import get_secret_str
CLAUDE_PLATFORM_SERVICE_NAME: Final = "aws-external-anthropic"
@ -15,6 +18,14 @@ def strip_claude_platform_route(model: str) -> str:
class BedrockClaudePlatformMixin(BaseAWSLLM):
def get_error_class(
self,
error_message: str,
status_code: int,
headers: dict[str, object] | httpx.Headers, # mutable-ok: base passes response headers as a dict
) -> BedrockError:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
@staticmethod
def _get_workspace_id(optional_params: dict, litellm_params: dict) -> str | None:
workspace_id = (

View file

@ -33,8 +33,53 @@ if TYPE_CHECKING:
from litellm.types.llms.openai import AllMessageValues
_ERROR_REQUEST_URL: Final = "https://docs.litellm.ai/docs"
def error_response_text(response: httpx.Response) -> str:
try:
return response.text
except httpx.ResponseNotRead:
return response.reason_phrase
def _synthesize_error_response(
*, status_code: int, headers: dict[str, object] | httpx.Headers, request: httpx.Request | None
) -> tuple[httpx.Request, httpx.Response]:
error_request: Final = request or httpx.Request(method="POST", url=_ERROR_REQUEST_URL)
safe_headers: Final = (
headers
if isinstance(headers, httpx.Headers)
else tuple((key, value) for key, value in headers.items() if isinstance(value, (str, bytes)))
)
return error_request, httpx.Response(status_code=status_code, headers=safe_headers, request=error_request)
class BedrockError(BaseLLMException):
pass
def __init__(
self,
status_code: int,
message: str,
headers: dict[str, object] | httpx.Headers | None = None,
request: httpx.Request | None = None,
response: httpx.Response | None = None,
body: dict[str, object] | None = None,
status_code_is_synthesized: bool = False,
) -> None:
error_request, error_response = (
_synthesize_error_response(status_code=status_code, headers=headers, request=request)
if response is None and headers
else (request, response)
)
super().__init__(
status_code=status_code,
message=message,
headers=headers,
request=error_request,
response=error_response,
body=body,
status_code_is_synthesized=status_code_is_synthesized,
)
_BEDROCK_AWS_AUTH_PARAMETER_KEYS: Final[tuple[str, ...]] = (

View file

@ -102,6 +102,8 @@ class BedrockCountTokensHandler(BedrockCountTokensConfig):
raise BedrockError(
status_code=response.status_code,
message=error_text,
headers=response.headers,
response=response,
)
bedrock_response: Final = response.json()
@ -124,6 +126,8 @@ class BedrockCountTokensHandler(BedrockCountTokensConfig):
raise BedrockError(
status_code=e.response.status_code,
message=e.response.text,
headers=e.response.headers,
response=e.response,
)
except Exception as e:
verbose_logger.error("Error in CountTokens handler: %s", e)

View file

@ -132,7 +132,12 @@ class BedrockEmbedding(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")
@ -161,7 +166,12 @@ class BedrockEmbedding(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")

View file

@ -20,6 +20,7 @@ import httpx
from litellm._logging import verbose_logger
from litellm.llms.base_llm.image_edit.transformation import BaseImageEditConfig
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.types.images.main import ImageEditOptionalRequestParams
from litellm.types.router import GenericLiteLLMParams
from litellm.types.utils import FileTypes, ImageObject, ImageResponse
@ -228,6 +229,14 @@ class BedrockAmazonNovaCanvasImageEditConfig(BaseImageEditConfig):
"""
return _supports_nova_canvas_image_edit_from_model_cost(model or "")
def get_error_class(
self,
error_message: str,
status_code: int,
headers: dict[str, object] | httpx.Headers, # mutable-ok: base passes response headers as a dict
) -> BedrockError:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def get_supported_openai_params(self, model: str) -> list:
return [
"n",

View file

@ -114,7 +114,12 @@ class BedrockImageEdit(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")
@ -156,7 +161,12 @@ class BedrockImageEdit(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")

View file

@ -27,6 +27,7 @@ from typing import TYPE_CHECKING, Any, Final
import httpx
from litellm.llms.base_llm.image_edit.transformation import BaseImageEditConfig
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.types.images.main import ImageEditOptionalRequestParams
from litellm.types.llms.stability import (
OPENAI_SIZE_TO_STABILITY_ASPECT_RATIO,
@ -84,6 +85,14 @@ class BedrockStabilityImageEditConfig(BaseImageEditConfig):
return True
return False
def get_error_class(
self,
error_message: str,
status_code: int,
headers: dict[str, object] | httpx.Headers, # mutable-ok: base passes response headers as a dict
) -> BedrockError:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def get_supported_openai_params(self, model: str) -> list:
"""
Return list of OpenAI params supported by Bedrock Stability.

View file

@ -119,7 +119,12 @@ class BedrockImageGeneration(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")
### FORMAT RESPONSE TO OPENAI FORMAT ###
@ -162,7 +167,12 @@ class BedrockImageGeneration(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")

View file

@ -29,6 +29,7 @@ from litellm.llms.bedrock.chat.invoke_transformations.base_invoke_transformation
AmazonInvokeConfig,
)
from litellm.llms.bedrock.common_utils import (
BedrockError,
apply_bedrock_invoke_structured_output,
ensure_bedrock_anthropic_messages_tool_names,
get_anthropic_beta_from_headers,
@ -79,6 +80,14 @@ class AmazonAnthropicClaudeMessagesConfig(
BEDROCK_INVOKE_ALLOWED_TOP_LEVEL_FIELDS = frozenset(BedrockInvokeAnthropicMessagesRequest.__annotations__.keys())
def get_error_class(
self,
error_message: str,
status_code: int,
headers: dict[str, object] | httpx.Headers, # mutable-ok: base passes response headers as a dict
) -> BedrockError:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def __init__(self, **kwargs):
BaseAnthropicMessagesConfig.__init__(self, **kwargs)
AmazonInvokeConfig.__init__(self, **kwargs)

View file

@ -2,13 +2,14 @@ import json
from collections.abc import Mapping
from typing import TYPE_CHECKING, Final, Optional, cast
import httpx
from httpx import Response
from litellm.litellm_core_utils.litellm_logging import Logging
from litellm.llms.base_llm.passthrough.transformation import BasePassthroughConfig
from ..base_aws_llm import BaseAWSLLM
from ..common_utils import BedrockEventStreamDecoderBase, BedrockModelInfo
from ..common_utils import BedrockError, BedrockEventStreamDecoderBase, BedrockModelInfo
if TYPE_CHECKING:
from httpx import URL
@ -18,6 +19,14 @@ if TYPE_CHECKING:
class BedrockPassthroughConfig(BaseAWSLLM, BedrockModelInfo, BedrockEventStreamDecoderBase, BasePassthroughConfig):
def get_error_class(
self,
error_message: str,
status_code: int,
headers: dict[str, object] | httpx.Headers, # mutable-ok: base passes response headers as a dict
) -> BedrockError:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def is_streaming_request(self, endpoint: str, request_data: dict) -> bool:
return "stream" in endpoint

View file

@ -9,12 +9,14 @@ import json
import uuid as uuid_lib
from typing import Final, cast
import httpx
from pydantic import BaseModel
from litellm._logging import verbose_logger
from litellm._uuid import uuid
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
from litellm.llms.base_llm.realtime.transformation import BaseRealtimeConfig
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.llms.bedrock.realtime.trigger_audio import ready_trigger_pcm
from litellm.types.llms.openai import (
OpenAIRealtimeContentPartDone,
@ -121,6 +123,14 @@ class BedrockRealtimeConfig(BaseRealtimeConfig):
self._cumulative_usage = BedrockUsageEvent()
self._reported_usage = BedrockUsageEvent()
def get_error_class(
self,
error_message: str,
status_code: int,
headers: dict[str, object] | httpx.Headers, # mutable-ok: base passes response headers as a dict
) -> BedrockError:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def validate_environment(self, headers: dict, model: str, api_key: str | None = None) -> dict:
"""Validate environment - no special validation needed for Bedrock."""
return headers

View file

@ -46,7 +46,12 @@ class BedrockRerankHandler(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")
@ -117,7 +122,12 @@ class BedrockRerankHandler(BaseAWSLLM):
response.raise_for_status()
except httpx.HTTPStatusError as err:
error_code: Final = err.response.status_code
raise BedrockError(status_code=error_code, message=err.response.text)
raise BedrockError(
status_code=error_code,
message=err.response.text,
headers=err.response.headers,
response=err.response,
)
except httpx.TimeoutException:
raise BedrockError(status_code=408, message="Timeout error occurred.")

View file

@ -39,7 +39,6 @@ from typing import Final
import httpx
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
from litellm.llms.base_llm.chat.transformation import BaseLLMException
from litellm.llms.base_llm.search.transformation import (
BaseSearchConfig,
SearchResponse,
@ -380,6 +379,7 @@ class AgentCoreSearchConfig(BaseSearchConfig, BaseAWSLLM):
raise BedrockError(
status_code=raw_response.status_code if raw_response.status_code >= 400 else 502,
message=f"AgentCore gateway MCP error: {error}",
headers=raw_response.headers,
)
# A failed tools/call is reported in-band, as HTTP 200 with result.isError
@ -389,6 +389,7 @@ class AgentCoreSearchConfig(BaseSearchConfig, BaseAWSLLM):
raise BedrockError(
status_code=raw_response.status_code if raw_response.status_code >= 400 else 502,
message=f"AgentCore web search tool error: {self._tool_error_message(response_json)}",
headers=raw_response.headers,
)
text_items: Final = tuple(
@ -440,6 +441,7 @@ class AgentCoreSearchConfig(BaseSearchConfig, BaseAWSLLM):
raise BedrockError(
status_code=502,
message=f"AgentCore gateway returned SSE without a JSON data frame: {text[:200]}",
headers=raw_response.headers,
)
def get_error_class(
@ -448,7 +450,7 @@ class AgentCoreSearchConfig(BaseSearchConfig, BaseAWSLLM):
status_code: int,
headers: dict, # mutable-ok: BaseSearchConfig.get_error_class takes the response headers as a dict
) -> Exception:
return BaseLLMException(
return BedrockError(
status_code=status_code,
message=error_message,
headers=headers,

View file

@ -8,6 +8,7 @@ from litellm._logging import verbose_logger
from litellm.litellm_core_utils.url_utils import encode_url_path_segment
from litellm.llms.base_llm.vector_store.transformation import BaseVectorStoreConfig
from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.types.integrations.rag.bedrock_knowledgebase import (
BedrockKBContent,
BedrockKBResponse,
@ -38,6 +39,14 @@ class BedrockVectorStoreConfig(BaseVectorStoreConfig, BaseAWSLLM):
BaseVectorStoreConfig.__init__(self)
BaseAWSLLM.__init__(self)
def get_error_class(
self,
error_message: str,
status_code: int,
headers: dict[str, object] | httpx.Headers, # mutable-ok: base passes response headers as a dict
) -> BedrockError:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def get_auth_credentials(self, litellm_params: dict) -> BaseVectorStoreAuthCredentials:
return {}

View file

@ -13,6 +13,8 @@ Auth: Bearer token (litellm_params.api_key, BEDROCK_MANTLE_API_KEY, or the
from collections.abc import AsyncIterator, Iterator
from typing import Any, Final
import httpx
import litellm
from litellm._logging import verbose_logger
from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM
@ -24,6 +26,8 @@ from litellm.secret_managers.main import get_secret_str
from litellm.types.llms.openai import AllMessageValues
from litellm.types.router import GenericLiteLLMParams
from ...base_llm.chat.transformation import BaseLLMException
from ...bedrock.common_utils import BedrockError
from ...openai_like.chat.transformation import OpenAILikeChatConfig
from ..common_utils import mantle_base_segment
@ -45,6 +49,11 @@ class BedrockMantleChatConfig(BedrockMantleAuthMixin, OpenAILikeChatConfig):
def get_config(cls):
return super().get_config()
def get_error_class(
self, error_message: str, status_code: int, headers: dict[str, object] | httpx.Headers
) -> BaseLLMException:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def _get_openai_compatible_provider_info(
self,
api_base: str | None,

View file

@ -19,11 +19,14 @@ import json
from collections.abc import Mapping
from typing import Any, Final
import httpx
from typing_extensions import ReadOnly, TypedDict
import litellm
from litellm._logging import verbose_logger
from litellm.llms.base_llm.chat.transformation import BaseLLMException
from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.llms.bedrock_mantle.common_utils import (
MANTLE_HOST_RE,
BedrockMantleAuthMixin,
@ -98,6 +101,11 @@ class BedrockMantleResponsesAPIConfig(BedrockMantleAuthMixin, OpenAIResponsesAPI
def custom_llm_provider(self) -> LlmProviders:
return LlmProviders.BEDROCK_MANTLE
def get_error_class(
self, error_message: str, status_code: int, headers: dict[str, object] | httpx.Headers
) -> BaseLLMException:
return BedrockError(status_code=status_code, message=error_message, headers=headers)
def get_complete_url(
self,
api_base: str | None,

View file

@ -3508,9 +3508,13 @@ class ProxyBaseLLMRequestProcessing:
error_body: Final = await http_status_error.response.aread()
error_text: Final = error_body.decode("utf-8")
error_headers: Final = { # mutable-ok: HTTPException takes a plain header dict
k: v if isinstance(v, str) else str(v) for k, v in safe_headers.items()
}
raise HTTPException(
status_code=http_status_error.response.status_code,
detail={"error": error_text},
headers=error_headers,
)
error_msg: Final = f"{e}"
# Check for AttributeError in the exception chain.

View file

@ -946,7 +946,14 @@ async def handle_bedrock_count_tokens(
except BedrockError as e:
# Convert BedrockError to HTTPException for FastAPI
verbose_proxy_logger.error("BedrockError in handle_bedrock_count_tokens: %s", e)
raise HTTPException(status_code=e.status_code, detail={"error": e.message})
from litellm.litellm_core_utils.llm_response_utils.get_headers import get_response_headers
provider_headers: Final = getattr(getattr(e, "response", None), "headers", None)
raise HTTPException(
status_code=e.status_code,
detail={"error": e.message},
headers=get_response_headers(provider_headers) if provider_headers else None,
)
except HTTPException:
# Re-raise HTTP exceptions as-is
raise

View file

@ -9,9 +9,11 @@ import litellm
from litellm.litellm_core_utils.exception_mapping_utils import (
ExceptionCheckers,
_get_body_error_code,
_get_response_headers,
exception_type,
extract_and_raise_litellm_exception,
)
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.llms.openai.common_utils import OpenAIError
from litellm.types.utils import LlmProviders
@ -1254,3 +1256,156 @@ def test_handle_error_marks_only_a_status_code_it_never_received():
raise handler._handle_error(e=upstream, provider_config=None)
assert received.value.status_code == 500
assert received.value.status_code_is_synthesized is False
def test_bedrock_500_preserves_provider_response_headers():
"""A Bedrock 5xx must keep x-amzn-RequestId so AWS support can trace it (LIT-5428)."""
provider_response = httpx.Response(
status_code=500,
headers={"x-amzn-RequestId": "req-map-500"},
text='{"message":"Amazon Bedrock is unable to process your request."}',
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
original_exception = BedrockError(
status_code=500,
message=provider_response.text,
headers=provider_response.headers,
response=provider_response,
)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
exception_type(
model="anthropic.claude-haiku-4-5-20251001-v1:0",
original_exception=original_exception,
custom_llm_provider="bedrock",
completion_kwargs={},
extra_kwargs={},
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-map-500"
@pytest.mark.parametrize(
"custom_llm_provider, status_code, provider_message, expected_exception",
[
(
"bedrock_mantle",
400,
(
'{"error":{"code":"validation_error",'
'"message":"prompt tokens (1055489) exceed model maximum (1050000) for openai.gpt-5.6-sol",'
'"param":null,"type":"invalid_request_error"}}'
),
litellm.ContextWindowExceededError,
),
(
"bedrock",
400,
'{"message":"Input is too long for requested model."}',
litellm.ContextWindowExceededError,
),
(
"bedrock",
400,
'{"message":"Could not process image"}',
litellm.InternalServerError,
),
],
)
def test_bedrock_classified_errors_preserve_provider_response_headers(
custom_llm_provider, status_code, provider_message, expected_exception
):
"""Branches that classify a Bedrock error by its text must keep x-amzn-RequestId (LIT-5428)."""
provider_response = httpx.Response(
status_code=status_code,
headers={"x-amzn-RequestId": "req-classified"},
text=provider_message,
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
original_exception = BedrockError(
status_code=status_code,
message=provider_message,
headers=provider_response.headers,
response=provider_response,
)
with pytest.raises(expected_exception) as exc_info:
exception_type(
model="anthropic.claude-haiku-4-5-20251001-v1:0",
original_exception=original_exception,
custom_llm_provider=custom_llm_provider,
completion_kwargs={},
extra_kwargs={},
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-classified"
@pytest.mark.parametrize(
"status_code, provider_message",
[
(504, '{"message":"Gateway timeout"}'),
(408, '{"message":"Bedrock did not answer in time"}'),
(408, '{"message":"Connect timeout on endpoint URL"}'),
],
)
def test_bedrock_timeout_mapping_preserves_provider_headers(status_code, provider_message):
"""A mapped bedrock timeout keeps the upstream response, like every other mapped bedrock error.
The proxy prefixes those headers on the way out, while retry and cooldown
logic still reads the raw retry-after off the response.
"""
provider_response = httpx.Response(
status_code=status_code,
headers={"x-amzn-RequestId": "req-timeout", "set-cookie": "session=attacker"},
text=provider_message,
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
original_exception = BedrockError(
status_code=status_code,
message=provider_message,
headers=provider_response.headers,
response=provider_response,
)
with pytest.raises(litellm.Timeout) as exc_info:
exception_type(
model="anthropic.claude-haiku-4-5-20251001-v1:0",
original_exception=original_exception,
custom_llm_provider="bedrock",
completion_kwargs={},
extra_kwargs={},
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-timeout"
assert exc_info.value.headers is None
@pytest.mark.parametrize("status_code", [504, 408])
def test_bedrock_timeout_mapping_keeps_retry_after_readable(status_code):
"""Cooldown and retry timing read retry-after through _get_response_headers."""
provider_response = httpx.Response(
status_code=status_code,
headers={"x-amzn-RequestId": "req-retry-after", "retry-after": "7"},
text='{"message":"Bedrock did not answer in time"}',
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
original_exception = BedrockError(
status_code=status_code,
message='{"message":"Bedrock did not answer in time"}',
headers=provider_response.headers,
response=provider_response,
)
with pytest.raises(litellm.Timeout) as exc_info:
exception_type(
model="anthropic.claude-haiku-4-5-20251001-v1:0",
original_exception=original_exception,
custom_llm_provider="bedrock",
completion_kwargs={},
extra_kwargs={},
)
exception_headers = _get_response_headers(original_exception=exc_info.value)
assert exception_headers is not None
assert litellm.utils._get_retry_after_from_exception_header(response_headers=exception_headers) == 7

View file

@ -177,3 +177,16 @@ def test_guardrail_config_flows_to_headers_not_request_body(model):
assert headers["X-Amzn-Bedrock-GuardrailIdentifier"] == "ff6ujrregl1q"
assert headers["X-Amzn-Bedrock-GuardrailVersion"] == "DRAFT"
assert headers["X-Amzn-Bedrock-Trace"] == "DISABLED"
def test_get_error_class_preserves_provider_headers():
"""The invoke handler path hands real provider headers to get_error_class (LIT-5428)."""
error = AmazonInvokeConfig().get_error_class(
error_message="Amazon Bedrock is unable to process your request.",
status_code=500,
headers={"x-amzn-RequestId": "req-invoke-500"},
)
assert isinstance(error, BedrockError)
assert error.headers == {"x-amzn-RequestId": "req-invoke-500"}
assert error.response.headers["x-amzn-requestid"] == "req-invoke-500"

View file

@ -2,6 +2,7 @@ import asyncio
import json
import os
import httpx
import pytest
from fastapi.testclient import TestClient
@ -6039,6 +6040,8 @@ def test_transform_response_does_not_leak_body_on_parse_failure():
leaky_body = {"output": {"message": {"content": [{"text": "secret content"}]}}}
class MockResponse:
headers = httpx.Headers({"x-amzn-RequestId": "req-parse-failure"})
def json(self):
return leaky_body
@ -6067,6 +6070,7 @@ def test_transform_response_does_not_leak_body_on_parse_failure():
msg = str(exc_info.value)
assert "secret content" not in msg
assert "Error converting to valid response block" in msg
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-parse-failure"
def test_converse_drops_sampling_params_for_models_that_removed_them():

View file

@ -496,3 +496,171 @@ async def test_async_invoke_streaming_forwards_bedrock_response_headers():
assert stream._hidden_params["additional_headers"]["llm_provider-x-amzn-requestid"] == "req-987"
def _bedrock_stream_error_response(status_code: int, request_id: str) -> httpx.Response:
return httpx.Response(
status_code=status_code,
headers={
"x-amzn-RequestId": request_id,
"x-amzn-ErrorType": "InternalServerException",
},
text='{"message":"Amazon Bedrock is unable to process your request."}',
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
def test_invoke_streaming_error_forwards_bedrock_response_headers():
error_response = _bedrock_stream_error_response(500, "req-stream-err-1")
client = HTTPHandler()
client.post = MagicMock(
side_effect=httpx.HTTPStatusError(
"server error",
request=error_response.request,
response=error_response,
)
)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
litellm.completion(
model="bedrock/invoke/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
stream=True,
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-stream-err-1"
@pytest.mark.asyncio
async def test_async_invoke_streaming_error_forwards_bedrock_response_headers():
error_response = _bedrock_stream_error_response(500, "req-stream-err-2")
client = AsyncHTTPHandler()
client.post = AsyncMock(
side_effect=httpx.HTTPStatusError(
"server error",
request=error_response.request,
response=error_response,
)
)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
await litellm.acompletion(
model="bedrock/invoke/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
stream=True,
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-stream-err-2"
def _unread_bedrock_stream_error_response(status_code: int, request_id: str) -> httpx.Response:
return httpx.Response(
status_code=status_code,
headers={
"x-amzn-RequestId": request_id,
"x-amzn-ErrorType": "InternalServerException",
},
stream=httpx.ByteStream(b'{"message":"Amazon Bedrock is unable to process your request."}'),
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
def test_invoke_streaming_error_forwards_headers_when_body_was_never_read():
"""A retried streamed request raises HTTPStatusError over a body nobody read, so
reading it for the error message throws and loses the request id (LIT-5428)."""
error_response = _unread_bedrock_stream_error_response(500, "req-unread-sync")
client = HTTPHandler()
client.post = MagicMock(
side_effect=httpx.HTTPStatusError(
"server error",
request=error_response.request,
response=error_response,
)
)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
litellm.completion(
model="bedrock/invoke/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
stream=True,
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-unread-sync"
@pytest.mark.asyncio
async def test_async_invoke_streaming_error_forwards_headers_when_body_was_never_read():
error_response = _unread_bedrock_stream_error_response(500, "req-unread-async")
client = AsyncHTTPHandler()
client.post = AsyncMock(
side_effect=httpx.HTTPStatusError(
"server error",
request=error_response.request,
response=error_response,
)
)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
await litellm.acompletion(
model="bedrock/invoke/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
stream=True,
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-unread-async"
def test_invoke_streaming_non_200_forwards_bedrock_response_headers():
"""A caller-supplied client that returns a failure instead of raising still reaches the
provider's headers, and reading the streamed body for the message must not throw (LIT-5428)."""
error_response = _unread_bedrock_stream_error_response(500, "req-non200-sync")
client = HTTPHandler()
client.post = MagicMock(return_value=error_response)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
litellm.completion(
model="bedrock/invoke/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
stream=True,
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-non200-sync"
@pytest.mark.asyncio
async def test_async_invoke_streaming_non_200_forwards_bedrock_response_headers():
error_response = _unread_bedrock_stream_error_response(500, "req-non200-async")
client = AsyncHTTPHandler()
client.post = AsyncMock(return_value=error_response)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
await litellm.acompletion(
model="bedrock/invoke/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
stream=True,
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-non200-async"

View file

@ -614,3 +614,260 @@ def test_sign_aws_request_assumes_role_with_external_id(monkeypatch):
authorization = {key.lower(): value for key, value in signed_headers.items()}["authorization"]
assert "ASIABATCHSIGNROLE" in authorization
assert signed_data == b'{"jobName": "litellm-batch-job"}'
# --------------------------------------------------------------------------- #
# Provider error headers (LIT-5428) #
# --------------------------------------------------------------------------- #
def _bedrock_chat_error_configs():
from litellm.llms.bedrock.chat.agentcore.transformation import AmazonAgentCoreConfig
from litellm.llms.bedrock.chat.converse_transformation import AmazonConverseConfig
from litellm.llms.bedrock.chat.invoke_agent.transformation import AmazonInvokeAgentConfig
from litellm.llms.bedrock.chat.invoke_transformations.amazon_moonshot_transformation import (
AmazonMoonshotConfig,
)
from litellm.llms.bedrock.chat.invoke_transformations.amazon_openai_transformation import (
AmazonBedrockOpenAIConfig,
)
from litellm.llms.bedrock.chat.invoke_transformations.base_invoke_transformation import (
AmazonInvokeConfig,
)
return [
AmazonInvokeConfig,
AmazonConverseConfig,
AmazonMoonshotConfig,
AmazonBedrockOpenAIConfig,
AmazonAgentCoreConfig,
AmazonInvokeAgentConfig,
]
@pytest.mark.parametrize("config", _bedrock_chat_error_configs())
def test_bedrock_chat_get_error_class_keeps_provider_headers(config):
"""Every Bedrock chat route must carry x-amzn-RequestId out to the caller (LIT-5428).
A config that drops the headers it is handed shadows the fix for its own models.
"""
error = config().get_error_class(
error_message="Amazon Bedrock is unable to process your request.",
status_code=500,
headers={"x-amzn-RequestId": "req-chat-500"},
)
assert error.response.headers["x-amzn-requestid"] == "req-chat-500"
def test_error_response_text_reads_a_read_response():
import httpx
from litellm.llms.bedrock.common_utils import error_response_text
response = httpx.Response(status_code=500, text="Amazon Bedrock is unable to process your request.")
assert error_response_text(response) == "Amazon Bedrock is unable to process your request."
def test_error_response_text_falls_back_when_a_streamed_response_was_never_read():
"""A retried streamed request raises HTTPStatusError over an unread body; reading it
throws ResponseNotRead and would lose the status and headers this fix preserves."""
import httpx
from litellm.llms.bedrock.common_utils import error_response_text
request = httpx.Request(method="POST", url="https://bedrock-runtime.amazonaws.com")
response = httpx.Response(
status_code=500,
headers={"x-amzn-RequestId": "req-unread-500"},
stream=httpx.ByteStream(b"never read"),
request=request,
)
with pytest.raises(httpx.ResponseNotRead):
_ = response.text
assert error_response_text(response) == "Internal Server Error"
def test_bedrock_error_skips_header_values_httpx_cannot_carry():
"""The shared HTTP handler copies an arbitrary exception's header values in verbatim,
so a non-str value must not take down the whole error (LIT-5428)."""
import httpx
from litellm.llms.bedrock.common_utils import BedrockError
error = BedrockError(
status_code=500,
message="boom",
headers={"x-amzn-RequestId": "req-mixed-500", "x-retry-count": 3, "x-nothing": None},
)
assert error.response.headers["x-amzn-requestid"] == "req-mixed-500"
assert "x-retry-count" not in error.response.headers
assert isinstance(error.response, httpx.Response)
def test_bedrock_error_keeps_duplicate_httpx_header_values():
import httpx
from litellm.llms.bedrock.common_utils import BedrockError
error = BedrockError(
status_code=500,
message="boom",
headers=httpx.Headers([("x-amzn-RequestId", "req-dup-500"), ("set-cookie", "a=1"), ("set-cookie", "b=2")]),
)
assert error.response.headers.get_list("set-cookie") == ["a=1", "b=2"]
def _bedrock_httpx_status_error_sites():
"""Every `except httpx.HTTPStatusError as err` that raises a BedrockError, across bedrock."""
import ast
import pathlib
sites = []
for path in sorted(pathlib.Path("litellm/llms/bedrock").rglob("*.py")):
tree = ast.parse(path.read_text())
for handler in (n for n in ast.walk(tree) if isinstance(n, ast.ExceptHandler)):
caught = ast.unparse(handler.type) if handler.type is not None else ""
if "HTTPStatusError" not in caught or handler.name is None:
continue
for call in (
n
for n in ast.walk(handler)
if isinstance(n, ast.Call) and isinstance(n.func, ast.Name) and n.func.id == "BedrockError"
):
sites.append((str(path), call.lineno, handler.name, {k.arg for k in call.keywords}))
return sites
def test_every_bedrock_httpx_status_error_site_keeps_provider_headers():
"""A raise site holding the provider's failed response must hand its headers on (LIT-5428).
These sites are the only place x-amzn-RequestId still exists; a site that drops it
silently shadows the fix for that whole surface.
"""
sites = _bedrock_httpx_status_error_sites()
assert len(sites) >= 12
dropped = [f"{path}:{lineno}" for path, lineno, _, kwargs in sites if "headers" not in kwargs]
assert dropped == []
@pytest.mark.parametrize("is_async", [False, True])
@pytest.mark.asyncio
async def test_bedrock_embedding_call_keeps_provider_headers(is_async):
"""The embeddings surface raises from the same shape as chat and lost the same header."""
import httpx
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.llms.bedrock.embed.embedding import BedrockEmbedding
from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler, HTTPHandler
failure = httpx.Response(
status_code=500,
headers={"x-amzn-RequestId": "req-embed-500"},
text='{"message":"Amazon Bedrock is unable to process your request."}',
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
class _SyncUpstream(HTTPHandler):
def post(self, *args, **kwargs):
return failure
class _AsyncUpstream(AsyncHTTPHandler):
async def post(self, *args, **kwargs):
return failure
async def _drive():
embedding = BedrockEmbedding()
kwargs = dict(
timeout=None,
api_base="https://bedrock-runtime.us-east-1.amazonaws.com/",
headers={},
data={},
)
if is_async:
return await embedding._make_async_call(client=_AsyncUpstream(), **kwargs)
return embedding._make_sync_call(client=_SyncUpstream(), **kwargs)
with pytest.raises(BedrockError) as exc_info:
await _drive()
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-embed-500"
def _bedrock_mantle_error_configs():
from litellm.llms.bedrock_mantle.chat.transformation import BedrockMantleChatConfig
from litellm.llms.bedrock_mantle.responses.transformation import BedrockMantleResponsesAPIConfig
return [BedrockMantleChatConfig, BedrockMantleResponsesAPIConfig]
@pytest.mark.parametrize("config", _bedrock_mantle_error_configs())
def test_bedrock_mantle_get_error_class_keeps_provider_headers(config):
"""bedrock_mantle rides the OpenAI-compatible surfaces, whose errors drop the headers.
A chat request for a responses-API model is bridged onto the responses config, so
fixing only the chat one leaves the model the customer actually calls uncovered.
"""
error = config().get_error_class(
error_message="prompt tokens exceed model maximum",
status_code=400,
headers={"x-amzn-RequestId": "req-mantle-400"},
)
assert error.response.headers["x-amzn-requestid"] == "req-mantle-400"
def _bedrock_configs_with_get_error_class():
import importlib
import inspect
import pathlib
import litellm
llms_root = pathlib.Path(inspect.getfile(litellm)).parent / "llms"
configs = []
for package in ("bedrock", "bedrock_mantle"):
for path in sorted((llms_root / package).rglob("*.py")):
module_name = "litellm.llms." + ".".join(path.relative_to(llms_root).with_suffix("").parts)
module = importlib.import_module(module_name)
for name, obj in vars(module).items():
if not inspect.isclass(obj) or obj.__module__ != module_name:
continue
if getattr(obj, "get_error_class", None) is None:
continue
configs.append(pytest.param(obj, id=f"{module_name}.{name}"))
return configs
@pytest.mark.parametrize("config", _bedrock_configs_with_get_error_class())
def test_every_bedrock_config_get_error_class_keeps_provider_headers(config):
"""Every bedrock surface must classify errors through BedrockError, not a header-dropping base.
A config that inherits get_error_class from a provider-agnostic base builds a blank
response, so the request id is gone before the proxy ever reads it.
"""
try:
instance = config()
except Exception:
instance = config.__new__(config)
try:
error = instance.get_error_class(
error_message="boom",
status_code=500,
headers={"x-amzn-RequestId": "req-audit-500"},
)
except Exception as raised: # some bases raise the exception instead of returning it
error = raised
assert error.response.headers["x-amzn-requestid"] == "req-audit-500"
def test_bedrock_get_error_class_audit_covers_every_surface():
assert len(_bedrock_configs_with_get_error_class()) >= 30

View file

@ -308,3 +308,64 @@ def test_completion_plumbs_stream_chunk_size_through_converse():
stream_chunk_size=2048,
)
iter_bytes_spy.assert_called_once_with(chunk_size=2048)
def _bedrock_error_response(status_code: int, request_id: str) -> httpx.Response:
return httpx.Response(
status_code=status_code,
headers={
"x-amzn-RequestId": request_id,
"x-amzn-ErrorType": "InternalServerException",
},
text=json.dumps({"message": "Amazon Bedrock is unable to process your request."}),
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
def test_converse_completion_error_forwards_bedrock_response_headers():
error_response = _bedrock_error_response(500, "req-err-123")
client = HTTPHandler()
client.post = MagicMock(
side_effect=httpx.HTTPStatusError(
"server error",
request=error_response.request,
response=error_response,
)
)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
litellm.completion(
model="bedrock/converse/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-err-123"
@pytest.mark.asyncio
async def test_async_converse_completion_error_forwards_bedrock_response_headers():
error_response = _bedrock_error_response(500, "req-err-456")
client = AsyncHTTPHandler()
client.post = AsyncMock(
side_effect=httpx.HTTPStatusError(
"server error",
request=error_response.request,
response=error_response,
)
)
with pytest.raises(litellm.ServiceUnavailableError) as exc_info:
await litellm.acompletion(
model="bedrock/converse/anthropic.claude-haiku-4-5-20251001-v1:0",
messages=[{"role": "user", "content": "hi"}],
client=client,
aws_access_key_id="fake",
aws_secret_access_key="fake",
aws_region_name="us-east-1",
)
assert exc_info.value.response.headers["x-amzn-requestid"] == "req-err-456"

View file

@ -5206,3 +5206,37 @@ class TestAzureRouterModelStreamingKeepalive:
assert result.headers["x-upstream"] == "kept"
assert chunks == [b"data: hello\n\n"]
@pytest.mark.asyncio
async def test_bedrock_count_tokens_error_forwards_provider_headers():
"""The count tokens route converts BedrockError into an HTTPException, and dropping the
headers there loses x-amzn-RequestId after the handler went to the trouble of keeping it."""
from fastapi import HTTPException
from litellm.llms.bedrock.common_utils import BedrockError
from litellm.proxy.pass_through_endpoints.llm_passthrough_endpoints import (
handle_bedrock_count_tokens,
)
failure = BedrockError(
status_code=500,
message="Amazon Bedrock is unable to process your request.",
headers={"x-amzn-RequestId": "req-count-tokens-500"},
)
with patch( # test-quality-ok: the route's BedrockError branch is only reachable when the handler raises
"litellm.llms.bedrock.count_tokens.handler.BedrockCountTokensHandler.handle_count_tokens_request",
new=AsyncMock(side_effect=failure),
):
with pytest.raises(HTTPException) as exc_info:
await handle_bedrock_count_tokens(
endpoint="v1/messages/count_tokens",
request=MagicMock(),
fastapi_response=MagicMock(),
user_api_key_dict=MagicMock(),
request_body={"model": "anthropic.claude-haiku-4-5-20251001-v1:0"},
)
assert exc_info.value.status_code == 500
assert exc_info.value.headers["llm_provider-x-amzn-requestid"] == "req-count-tokens-500"

View file

@ -8259,3 +8259,38 @@ class TestPassthroughHeadersAcceptImmutableMappings:
assert merged["content-type"] == "text/event-stream"
# the excluded hop-by-hop header is still dropped
assert "transfer-encoding" not in merged
@pytest.mark.asyncio
async def test_handle_llm_api_exception_forwards_provider_headers_on_http_status_error():
"""The httpx.HTTPStatusError branch dropped the headers its sibling branches forward.
A Bedrock passthrough failure reaches this branch, so the request id was gone
before the client saw the response.
"""
import httpx
from litellm.proxy._types import UserAPIKeyAuth
request = httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/model/m/converse")
response = httpx.Response(
status_code=500,
headers={"x-amzn-RequestId": "req-passthrough-500"},
content=b'{"message": "Amazon Bedrock is unable to process your request."}',
request=request,
)
processor = ProxyBaseLLMRequestProcessing(data={})
proxy_logging_obj = MagicMock()
proxy_logging_obj.post_call_failure_hook = AsyncMock(return_value=None)
proxy_logging_obj.post_call_response_headers_hook = AsyncMock(return_value={})
with pytest.raises(HTTPException) as exc_info:
await processor._handle_llm_api_exception(
e=httpx.HTTPStatusError("boom", request=request, response=response),
user_api_key_dict=UserAPIKeyAuth(api_key="sk-test"),
proxy_logging_obj=proxy_logging_obj,
)
assert exc_info.value.headers is not None
assert exc_info.value.headers["llm_provider-x-amzn-requestid"] == "req-passthrough-500"

View file

@ -18,6 +18,7 @@ from litellm.exceptions import (
ImageFetchError,
MidStreamFallbackError,
RateLimitError,
ServiceUnavailableError,
)
@ -312,3 +313,85 @@ class TestProxyHeaderExtraction:
# Verify headers are extracted and prefixed correctly
assert headers.get("llm_provider-x-request-id") == "req-abc123"
assert headers.get("llm_provider-x-ms-region") == "eastus"
class TestBedrockErrorHeaders:
"""A BedrockError built with headers but no response still exposes them (LIT-5428)."""
def test_synthesized_response_carries_headers(self):
from litellm.llms.bedrock.common_utils import BedrockError
error = BedrockError(
status_code=500,
message="Amazon Bedrock is unable to process your request.",
headers={"x-amzn-RequestId": "req-base-500"},
)
assert error.response.headers["x-amzn-requestid"] == "req-base-500"
assert str(error.request.url) == str(BedrockError(status_code=500, message="boom").request.url)
assert str(error.response.request.url) == str(error.request.url)
def test_synthesized_response_without_headers_stays_empty(self):
from litellm.llms.bedrock.common_utils import BedrockError
error = BedrockError(status_code=500, message="boom")
assert dict(error.response.headers) == {}
def test_explicit_response_is_kept(self):
from litellm.llms.bedrock.common_utils import BedrockError
provider_response = httpx.Response(
status_code=500,
headers={"x-amzn-RequestId": "from-response"},
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
error = BedrockError(
status_code=500,
message="boom",
headers={"x-amzn-RequestId": "from-headers"},
response=provider_response,
)
assert error.response is provider_response
def test_proxy_extraction_surfaces_bedrock_request_id(self):
"""End-to-end shape the proxy error handler returns to the caller."""
from litellm.litellm_core_utils.exception_mapping_utils import exception_type
from litellm.litellm_core_utils.llm_response_utils.get_headers import (
get_response_headers,
)
from litellm.llms.bedrock.common_utils import BedrockError
provider_response = httpx.Response(
status_code=500,
headers={"x-amzn-RequestId": "req-proxy-500"},
text='{"message":"Amazon Bedrock is unable to process your request."}',
request=httpx.Request("POST", "https://bedrock-runtime.us-east-1.amazonaws.com/"),
)
with pytest.raises(ServiceUnavailableError) as exc_info:
exception_type(
model="anthropic.claude-haiku-4-5-20251001-v1:0",
original_exception=BedrockError(
status_code=500,
message=provider_response.text,
headers=provider_response.headers,
response=provider_response,
),
custom_llm_provider="bedrock",
completion_kwargs={},
extra_kwargs={},
)
# Mirrors ProxyBaseLLMRequestProcessing._handle_llm_api_exception
error = exc_info.value
headers = getattr(error, "headers", None) or {}
if not headers:
_response = getattr(error, "response", None)
if _response is not None:
_response_headers = getattr(_response, "headers", None)
if _response_headers:
headers = get_response_headers(dict(_response_headers))
assert headers.get("llm_provider-x-amzn-requestid") == "req-proxy-500"