Implements the Bedrock InvokeModel request shaping that returned 501: anthropic_version, the top-level allowlist, model/stream moved to the URL, header and feature betas in the body's anthropic_beta, clear_thinking, cache_control, structured output and tool fixes in Python's order. Streams decode the AWS event stream into Anthropic SSE. genconfig and Terraform now default to half Bedrock deployments signed with SigV4.
Add the openai provider to /v1/chat/completions. The caller's body is passed
through with only the model renamed and LiteLLM-owned keys removed, and the
provider's JSON or SSE bytes are relayed as received. Streams have no total
timeout, only a per-read idle limit, so 30+ minute generations are not cut off.
openai_like and the translated providers keep their existing path.
Adds litellm-rust/loadtest: a mock LLM (OpenAI chat, Anthropic messages, Bedrock invoke with
event-stream framing) that takes its timing from the prompt, an open-loop load generator with
workloads covering 10k to 1M token contexts, 6 minute non-streaming calls and 30 minute streams,
a config and virtual key generator, a summarizer and a /proc sampler, plus Terraform for an AWS
rig with load generators, gateway and mocks on separate machines
The gateway sets TCP_NODELAY on accepted sockets, removing a ~40 ms stall per streamed response,
and its Docker build takes LTO and CODEGEN_UNITS build args for faster iteration builds
