mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-05 08:07:05 +00:00
445c67cfec
This commit adds a complete 1-click deployment solution for LiteLLM on AWS ECS, configured to match the benchmark specifications from https://docs.litellm.ai/docs/benchmarks ## What's Added ### Infrastructure (1 file) - cloudformation-ecs.yaml: AWS CloudFormation template for ECS deployment - 4 ECS Fargate tasks (4 vCPU, 8 GB RAM each) - 4 workers per task (16 total workers) - RDS PostgreSQL database (db.t3.medium) - Application Load Balancer - VPC with public/private subnets across 2 AZs - Security groups, NAT Gateway, monitoring ### Deployment Tools (3 files) - deploy.sh: Automated deployment script with interactive prompts - test-deployment.sh: Deployment validation and health check script - cost-calculator.sh: Interactive cost estimation tool ### Documentation (6 files) - 00-START-HERE.md: Quick start guide and overview - QUICKSTART.md: 5-minute deployment guide - README.md: Complete deployment documentation - ARCHITECTURE.md: Detailed architecture deep-dive with diagrams - INDEX.md: Master index of all files - .summary.md: Internal summary document ### Testing & Configuration (2 files) - locustfile.py: Load testing script to replicate benchmark tests - example-config.yaml: LiteLLM configuration example ## Configuration - 4 instances with 4 vCPU and 8 GB RAM each - 4 workers per instance - Expected performance: - Median latency: ~100 ms - P95 latency: ~150 ms - Throughput: ~1,170 RPS - LiteLLM overhead: ~2 ms ## Usage ```bash cd deploy/aws ./deploy.sh ``` ## Monthly Cost ~$440-460 (pay-as-you-go) or ~$270-370 (with reserved capacity) ## Features - ✅ CloudFormation template validated with AWS - ✅ Production-ready with high availability - ✅ Secure by default (private subnets, security groups, encrypted secrets) - ✅ Well-documented with comprehensive guides - ✅ Includes validation and load testing tools - ✅ Cost-optimized configuration Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| aws | ||
| azure_resource_manager | ||
| charts/litellm-helm | ||
| kubernetes | ||
| Dockerfile.ghcr_base | ||