Adds litellm-rust/loadtest: a mock LLM (OpenAI chat, Anthropic messages, Bedrock invoke with
event-stream framing) that takes its timing from the prompt, an open-loop load generator with
workloads covering 10k to 1M token contexts, 6 minute non-streaming calls and 30 minute streams,
a config and virtual key generator, a summarizer and a /proc sampler, plus Terraform for an AWS
rig with load generators, gateway and mocks on separate machines
The gateway sets TCP_NODELAY on accepted sockets, removing a ~40 ms stall per streamed response,
and its Docker build takes LTO and CODEGEN_UNITS build args for faster iteration builds
d93353bcb6
