mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-05 08:07:05 +00:00
1cf0097be6
After testing the AWS ECS deployment, discovered that the CloudFormation template does not configure the fake-openai-endpoint model required for running benchmark tests as documented in https://docs.litellm.ai/docs/benchmarks ## Changes - Added KNOWN_ISSUES.md documenting the limitation in detail - Updated README.md with prominent warning about the gap - Updated 00-START-HERE.md with limitation notice - Added simple-loadtest.py for infrastructure-only testing ## What Was Tested ✅ Successfully deployed: - 4 ECS Fargate tasks (4 vCPU, 8 GB RAM each) - RDS PostgreSQL database - Application Load Balancer - Full VPC with security groups ✅ Verified working: - All tasks running and healthy - Database connections - Health endpoints responding - Load balancer routing ❌ Cannot test (missing model config): - API /v1/chat/completions requests - Locust benchmark with 1000 users - LiteLLM overhead measurement - Performance metrics (latency, RPS) ## Root Cause The CloudFormation template only sets environment variables but does not: - Mount a config.yaml file - Configure model_list with fake-openai-endpoint - Set up the test endpoint needed for benchmarking ## Impact Users can deploy the infrastructure matching benchmark specs, but cannot run the actual benchmark without manually configuring models via API or updating the template to mount a config file. ## Stack Cleanup The test deployment was deleted to avoid ongoing costs (~$440/month). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| aws | ||
| azure_resource_manager | ||
| charts/litellm-helm | ||
| kubernetes | ||
| Dockerfile.ghcr_base | ||