scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-12 23:01:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88

feat(terraform/aws): make VPC, Aurora, and Redis optional

Browse source 
Adds vpc_id/public_subnet_ids/private_subnet_ids to deploy into existing networking, plus create_database/database_url and create_redis/redis_url to use existing data stores or none at all. Defaults keep today's module-owned behavior.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
yassin 2026-08-12 02:25:48 +00:00
parent 06943b6468
commit 8fe7bcdcd7
18 changed files with 758 additions and 123 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

54
.github/workflows/test-terraform-modules.yml vendored Normal file
View file

@ -0,0 +1,54 @@
name: Terraform Modules
on:
push:
paths:
- "terraform/litellm/**"
- ".github/workflows/test-terraform-modules.yml"
pull_request:
branches:
- main
- litellm_internal_staging
- litellm_oss_staging
- "litellm_**"
paths:
- "terraform/litellm/**"
- ".github/workflows/test-terraform-modules.yml"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
aws-module:
name: fmt, validate, test (aws)
runs-on: ubuntu-latest
timeout-minutes: 15
defaults:
run:
working-directory: terraform/litellm/aws
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
persist-credentials: false
- uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
with:
terraform_version: 1.13.3
terraform_wrapper: false
- name: fmt
run: terraform fmt -recursive -check -diff
- name: init
run: terraform init -backend=false -input=false
- name: validate
run: terraform validate
# Plan-only, mock_provider-backed: no AWS credentials, no API calls.
- name: test
run: terraform test

63
terraform/litellm/aws/README.md
View file

@ -2,9 +2,9 @@
Deploys the componentized LiteLLM proxy on AWS:
- **VPC** with public + private subnets across the AZs you pass in, one NAT gateway
- **Aurora Postgres** cluster — one writer instance + one reader instance, **IAM database authentication enabled**
- **ElastiCache Redis** (private, replication group with multi-AZ failover and at-rest + in-transit encryption) for caching + rate limiting
- **VPC** with public + private subnets across the AZs you pass in, one NAT gateway (skipped when you pass an existing `vpc_id`)
- **Aurora Postgres** cluster — one writer instance + one reader instance, **IAM database authentication enabled** (skipped when `create_database = false`)
- **ElastiCache Redis** (private, replication group with multi-AZ failover and at-rest + in-transit encryption) for caching + rate limiting (skipped when `create_redis = false`)
- **S3 bucket** (private, versioned, SSE-S3) — exposed to gateway + backend as `S3_BUCKET_NAME` / `S3_REGION_NAME` for cache backend, request log archival, and `/v1/files` storage
- **Secrets Manager** entries for `LITELLM_MASTER_KEY` (auto-generated, `sk-…`) and the Aurora master password (bootstrap-only)
- **ECS Fargate cluster** running three services — `gateway`, `backend`, `ui`
@ -14,6 +14,58 @@ Deploys the componentized LiteLLM proxy on AWS:
- Everything else (management API: `/key/*`, `/user/*`, …) → `backend`
- **One-off migration task** (`litellm-migrations`) that runs `prisma migrate deploy` from the dedicated `ghcr.io/berriai/litellm-migrations` image
## Bring your own networking, database, and Redis
The three infrastructure pieces the stack would otherwise own are each
optional, so it can slot into an account where networking and data stores are
already provisioned (often by another team, in another Terraform state).
**Networking.** Set `vpc_id` plus `public_subnet_ids` and `private_subnet_ids`
and no VPC, subnet, route table, internet gateway, or NAT gateway is created.
The ALB goes in the public subnets, the ECS tasks and any subnet group the
stack still needs go in the private ones, and `vpc_cidr` / `azs` go unused.
The private subnets need their own egress (NAT gateway, or VPC endpoints
covering ECR, S3, CloudWatch Logs, and Secrets Manager) since tasks pull
images, resolve secrets, and call LLM providers.
Security groups stay module-owned in either mode: the ALB group, the tasks
group, and the database/cache groups when it creates those. To let the tasks
reach infrastructure the module doesn't manage, either allow inbound from the
group named by the `task_security_group_id` output, or attach a group of your
own with `additional_task_security_group_ids`.
```hcl
vpc_id = "vpc-0123456789abcdef0"
public_subnet_ids = ["subnet-aaa", "subnet-bbb"]
private_subnet_ids = ["subnet-ccc", "subnet-ddd"]
```
**Database and Redis.** `create_database` and `create_redis` default to `true`
(today's behavior). Set one to `false` and pass a connection string to use
something you already run: the value lands in a Secrets Manager entry and
reaches gateway, backend, and the migration task as `DATABASE_URL` /
`REDIS_URL`, both of which outrank the discrete `DATABASE_*` / `REDIS_*` vars
in the proxy, so nothing appears in plain text in a task definition.
```hcl
create_database = false
database_url = "postgresql://litellm:...@db.internal:5432/litellm"
create_redis = false
redis_url = "rediss://:...@cache.internal:6379"
```
The schema migration still runs on every apply against an existing database;
only the Aurora-specific IAM-user bootstrap drops out, since those credentials
are already in the URL.
Leaving the URL empty runs without the component entirely:
- No database: no virtual keys, teams, spend tracking, or UI persistence, and
`STORE_MODEL_IN_DB` is not set, so models come from `proxy_config`. Requests
authenticate with `LITELLM_MASTER_KEY` only.
- No Redis: rate limits, budgets, and router cooldowns are per-task rather
than cluster-wide, which is only sane at one task per service.
## Aurora + IAM auth
The cluster runs with `iam_database_authentication_enabled = true`. Enabling
@ -354,6 +406,9 @@ Three opt-in tripwires guard against accidental data loss on
`/v1/files` content, and the S3 cache backend; default `false`) —
`terraform destroy` against a non-empty bucket fails.
Neither applies to a database you brought yourself: its lifecycle stays with
whoever provisioned it, and `terraform destroy` leaves it alone.
Flip either to `true` only for ephemeral / CI stacks where you accept
losing the contents.
@ -365,7 +420,7 @@ losing the contents.
| `examples/default/` | Thin root: `aws` provider (with an optional `default_tags` slot for org-wide tags) + a call to the module. The one-command deploy path. |
| `variables.tf` | All input variables |
| `locals.tf` | Path-prefix lists for ALB routing (mirror of `helm/.../ingress.yaml`) |
| `network.tf` | VPC, subnets, IGW, NAT, route tables, security groups |
| `network.tf` | VPC, subnets, IGW, NAT, route tables (all optional), security groups |
| `secrets.tf` | Secrets Manager entries + random passwords |
| `rds.tf` | Aurora Postgres cluster + writer / reader instances |
| `redis.tf` | ElastiCache Redis |

15
terraform/litellm/aws/alb.tf
View file

@ -3,10 +3,17 @@ resource "aws_lb" "this" {
load_balancer_type = "application"
internal = false
security_groups = [aws_security_group.alb.id]
subnets = aws_subnet.public[*].id
subnets = local.public_subnet_ids
idle_timeout = 120
lifecycle {
precondition {
condition = length(local.public_subnet_ids) >= 2
error_message = "The ALB needs at least 2 public subnets in different AZs. Set `public_subnet_ids` when using `vpc_id`, or list at least 2 `azs` when the module creates the VPC."
}
}
tags = local.tags
}
@ -25,7 +32,7 @@ resource "aws_lb_target_group" "gateway" {
port = 4000
protocol = "HTTP"
target_type = "ip"
vpc_id = aws_vpc.this.id
vpc_id = local.vpc_id
health_check {
path = "/health/readiness"
@ -46,7 +53,7 @@ resource "aws_lb_target_group" "backend" {
port = 4001
protocol = "HTTP"
target_type = "ip"
vpc_id = aws_vpc.this.id
vpc_id = local.vpc_id
health_check {
path = "/health/readiness"
@ -67,7 +74,7 @@ resource "aws_lb_target_group" "ui" {
port = 3000
protocol = "HTTP"
target_type = "ip"
vpc_id = aws_vpc.this.id
vpc_id = local.vpc_id
health_check {
path = "/healthz"

53
terraform/litellm/aws/bootstrap.tf
View file

@ -1,9 +1,12 @@
# Auto-runs the two manual steps that used to follow `terraform apply`:
#
# 1. Create the IAM-authed Postgres user (litellm_app) — uses the postgres:16
# image with the master password from Secrets Manager.
# image with the master password from Secrets Manager. Only relevant to
# the Aurora cluster this module creates, so it is skipped when
# create_database = false.
# 2. Run prisma migrate deploy — reuses the existing aws_ecs_task_definition
# .migrations task def from migrations.tf.
# .migrations task def from migrations.tf. Runs against an existing
# database too, and only disappears when there is no database at all.
#
# Both are invoked via `terraform_data` provisioners. Gateway/backend services
# in ecs.tf depend on `terraform_data.migration`, so on a fresh apply they
@ -23,13 +26,14 @@
# extras — see iam.tf). The DB master password lives in a separate secret used
# only here, so we grant access in an additive policy.
resource "aws_iam_policy" "bootstrap_secrets" {
name = "${local.name}-bootstrap-secrets-access"
count = var.create_database ? 1 : 0
name = "${local.name}-bootstrap-secrets-access"
policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Action = ["secretsmanager:GetSecretValue"]
Resource = [aws_secretsmanager_secret.db_master_password.arn]
Resource = [aws_secretsmanager_secret.db_master_password[0].arn]
}]
})
@ -37,12 +41,14 @@ resource "aws_iam_policy" "bootstrap_secrets" {
}
resource "aws_iam_role_policy_attachment" "task_execution_bootstrap_secrets" {
count = var.create_database ? 1 : 0
role = aws_iam_role.task_execution.name
policy_arn = aws_iam_policy.bootstrap_secrets.arn
policy_arn = aws_iam_policy.bootstrap_secrets[0].arn
}
# ---------- Bootstrap task def ----------
resource "aws_cloudwatch_log_group" "bootstrap_db" {
count = var.create_database ? 1 : 0
name = "/ecs/${local.name}/bootstrap-db"
retention_in_days = var.log_retention_days
@ -68,6 +74,7 @@ locals {
}
resource "aws_ecs_task_definition" "bootstrap_db" {
count = var.create_database ? 1 : 0
family = "${local.name}-bootstrap-db"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
@ -82,15 +89,15 @@ resource "aws_ecs_task_definition" "bootstrap_db" {
essential = true
environment = [
{ name = "PGHOST", value = aws_rds_cluster.this.endpoint },
{ name = "PGPORT", value = tostring(aws_rds_cluster.this.port) },
{ name = "PGHOST", value = aws_rds_cluster.this[0].endpoint },
{ name = "PGPORT", value = tostring(aws_rds_cluster.this[0].port) },
{ name = "PGUSER", value = var.db_master_username },
{ name = "PGDATABASE", value = var.db_name },
{ name = "BOOTSTRAP_SQL", value = local.bootstrap_sql },
]
secrets = [
# `:password::` extracts the password field out of the JSON secret.
{ name = "PGPASSWORD", valueFrom = "${aws_secretsmanager_secret.db_master_password.arn}:password::" },
{ name = "PGPASSWORD", valueFrom = "${aws_secretsmanager_secret.db_master_password[0].arn}:password::" },
]
entryPoint = ["sh", "-c"]
@ -99,7 +106,7 @@ resource "aws_ecs_task_definition" "bootstrap_db" {
logConfiguration = {
logDriver = "awslogs"
options = {
awslogs-group = aws_cloudwatch_log_group.bootstrap_db.name
awslogs-group = aws_cloudwatch_log_group.bootstrap_db[0].name
awslogs-region = var.region
awslogs-stream-prefix = "bootstrap"
}
@ -111,20 +118,22 @@ resource "aws_ecs_task_definition" "bootstrap_db" {
# ---------- Bootstrap trigger ----------
resource "terraform_data" "bootstrap_db" {
count = var.create_database ? 1 : 0
triggers_replace = {
cluster_resource_id = aws_rds_cluster.this.cluster_resource_id
task_def_revision = aws_ecs_task_definition.bootstrap_db.revision
cluster_resource_id = aws_rds_cluster.this[0].cluster_resource_id
task_def_revision = aws_ecs_task_definition.bootstrap_db[0].revision
}
provisioner "local-exec" {
interpreter = ["bash", "-c"]
environment = {
CLUSTER = aws_ecs_cluster.this.name
TASK_DEF = aws_ecs_task_definition.bootstrap_db.arn
SUBNETS = join(",", aws_subnet.private[*].id)
SG = aws_security_group.tasks.id
TASK_DEF = aws_ecs_task_definition.bootstrap_db[0].arn
SUBNETS = join(",", local.private_subnet_ids)
SG = join(",", local.task_security_group_ids)
REGION = var.region
LOG_GRP = aws_cloudwatch_log_group.bootstrap_db.name
LOG_GRP = aws_cloudwatch_log_group.bootstrap_db[0].name
}
command = <<-EOT
set -euo pipefail
@ -154,20 +163,22 @@ resource "terraform_data" "bootstrap_db" {
# Reuses the task definition from migrations.tf — this resource just invokes
# it and waits.
resource "terraform_data" "migration" {
count = local.database_enabled ? 1 : 0
triggers_replace = {
task_def_revision = aws_ecs_task_definition.migrations.revision
bootstrap_id = terraform_data.bootstrap_db.id
task_def_revision = aws_ecs_task_definition.migrations[0].revision
bootstrap_id = join(",", terraform_data.bootstrap_db[*].id)
}
provisioner "local-exec" {
interpreter = ["bash", "-c"]
environment = {
CLUSTER = aws_ecs_cluster.this.name
TASK_DEF = aws_ecs_task_definition.migrations.arn
SUBNETS = join(",", aws_subnet.private[*].id)
SG = aws_security_group.tasks.id
TASK_DEF = aws_ecs_task_definition.migrations[0].arn
SUBNETS = join(",", local.private_subnet_ids)
SG = join(",", local.task_security_group_ids)
REGION = var.region
LOG_GRP = aws_cloudwatch_log_group.migrations.name
LOG_GRP = aws_cloudwatch_log_group.migrations[0].name
}
command = <<-EOT
set -euo pipefail

82
terraform/litellm/aws/ecs.tf
View file

@ -31,6 +31,7 @@ resource "aws_cloudwatch_log_group" "ui" {
}
resource "aws_cloudwatch_log_group" "migrations" {
count = local.database_enabled ? 1 : 0
name = "/ecs/${local.name}/migrations"
retention_in_days = var.log_retention_days
@ -38,11 +39,13 @@ resource "aws_cloudwatch_log_group" "migrations" {
}
# Shared env block fed to gateway, backend, and the migration task. Mirrors
# the helm chart's `litellm.serverEnv` helper on the IAM-auth branch:
# DATABASE_URL is assembled at runtime by
# the helm chart's `litellm.serverEnv` helper on the IAM-auth branch: for the
# module-created Aurora, DATABASE_URL is assembled at runtime by
# litellm/proxy/auth/rds_iam_token.py::init_iam_db_url_from_env from
# HOST/PORT/USER/NAME plus an IAM-signed token, so no DB password is needed
# in the task definition.
# in the task definition. An existing database instead arrives as a
# DATABASE_URL secret (var.database_url), which run.py and the proxy both
# take as-is.
locals {
# OTel v2 is opt-in and gated on otel_endpoint, matching the GCP stack.
# When set, LITELLM_OTEL_V2 flips on alongside the OTEL_* block, with
@ -103,29 +106,50 @@ locals {
] : [],
)
shared_env = [
managed_db_env = var.create_database ? [
{ name = "IAM_TOKEN_DB_AUTH", value = "true" },
{ name = "DATABASE_HOST", value = aws_rds_cluster.this.endpoint },
{ name = "DATABASE_PORT", value = tostring(aws_rds_cluster.this.port) },
{ name = "DATABASE_HOST", value = aws_rds_cluster.this[0].endpoint },
{ name = "DATABASE_PORT", value = tostring(aws_rds_cluster.this[0].port) },
{ name = "DATABASE_USER", value = var.db_username },
{ name = "DATABASE_NAME", value = var.db_name },
{ name = "DATABASE_HOST_READ_REPLICA", value = aws_rds_cluster.this.reader_endpoint },
{ name = "DATABASE_PORT_READ_REPLICA", value = tostring(aws_rds_cluster.this.port) },
{ name = "REDIS_HOST", value = aws_elasticache_replication_group.this.primary_endpoint_address },
{ name = "REDIS_PORT", value = tostring(aws_elasticache_replication_group.this.port) },
{ name = "DATABASE_HOST_READ_REPLICA", value = aws_rds_cluster.this[0].reader_endpoint },
{ name = "DATABASE_PORT_READ_REPLICA", value = tostring(aws_rds_cluster.this[0].port) },
] : []
managed_redis_env = var.create_redis ? [
{ name = "REDIS_HOST", value = aws_elasticache_replication_group.this[0].primary_endpoint_address },
{ name = "REDIS_PORT", value = tostring(aws_elasticache_replication_group.this[0].port) },
# transit_encryption_enabled = true on the replication group means the
# proxy must connect via rediss://. _redis.get_redis_url_from_environment
# honors REDIS_SSL to flip the scheme.
{ name = "REDIS_SSL", value = "true" },
# S3 bucket — referenced from proxy_config via os.environ/S3_BUCKET_NAME
# (e.g. cache backend, request log archival, /files passthrough).
{ name = "S3_BUCKET_NAME", value = aws_s3_bucket.this.bucket },
{ name = "S3_REGION_NAME", value = var.region },
# boto3 inside generate_iam_auth_token reads AWS_REGION_NAME first, then
# AWS_REGION. Set both for compatibility.
{ name = "AWS_REGION", value = var.region },
{ name = "AWS_REGION_NAME", value = var.region },
]
] : []
shared_env = concat(
local.managed_db_env,
local.managed_redis_env,
[
# S3 bucket — referenced from proxy_config via os.environ/S3_BUCKET_NAME
# (e.g. cache backend, request log archival, /files passthrough).
{ name = "S3_BUCKET_NAME", value = aws_s3_bucket.this.bucket },
{ name = "S3_REGION_NAME", value = var.region },
# boto3 inside generate_iam_auth_token reads AWS_REGION_NAME first, then
# AWS_REGION. Set both for compatibility.
{ name = "AWS_REGION", value = var.region },
{ name = "AWS_REGION_NAME", value = var.region },
],
)
# DATABASE_URL / REDIS_URL both outrank the discrete host/port vars in the
# proxy, so the BYO branch needs nothing removed from shared_env: the
# managed_*_env blocks are already empty whenever these are set.
byo_database_secrets = local.byo_database ? [
{ name = "DATABASE_URL", valueFrom = aws_secretsmanager_secret.database_url[0].arn },
] : []
byo_redis_secrets = local.byo_redis ? [
{ name = "REDIS_URL", valueFrom = aws_secretsmanager_secret.redis_url[0].arn },
] : []
shared_secrets = concat(
[
@ -134,6 +158,8 @@ locals {
var.litellm_license == "" ? [] : [
{ name = "LITELLM_LICENSE", valueFrom = aws_secretsmanager_secret.license[0].arn },
],
local.byo_database_secrets,
local.byo_redis_secrets,
local.otel_secrets,
local.billing_metrics_secrets,
)
@ -151,9 +177,11 @@ locals {
for k, v in var.backend_extra_env : { name = k, value = v }
]
backend_default_env = [
# Storing models in the DB needs a DB. Without one the backend reads its
# model list from proxy_config only.
backend_default_env = local.database_enabled ? [
{ name = "STORE_MODEL_IN_DB", value = "true" },
]
] : []
gateway_extra_secrets_list = [
for k, v in var.gateway_extra_secrets : { name = k, valueFrom = v }
]
@ -286,8 +314,8 @@ resource "aws_ecs_service" "gateway" {
launch_type = "FARGATE"
network_configuration {
subnets = aws_subnet.private[*].id
security_groups = [aws_security_group.tasks.id]
subnets = local.private_subnet_ids
security_groups = local.task_security_group_ids
assign_public_ip = false
}
@ -381,8 +409,8 @@ resource "aws_ecs_service" "backend" {
launch_type = "FARGATE"
network_configuration {
subnets = aws_subnet.private[*].id
security_groups = [aws_security_group.tasks.id]
subnets = local.private_subnet_ids
security_groups = local.task_security_group_ids
assign_public_ip = false
}
@ -451,8 +479,8 @@ resource "aws_ecs_service" "ui" {
launch_type = "FARGATE"
network_configuration {
subnets = aws_subnet.private[*].id
security_groups = [aws_security_group.tasks.id]
subnets = local.private_subnet_ids
security_groups = local.task_security_group_ids
assign_public_ip = false
}

9
terraform/litellm/aws/examples/default/main.tf
View file

@ -24,6 +24,15 @@ module "litellm" {
env = var.env
azs = var.azs
vpc_id = var.vpc_id
public_subnet_ids = var.public_subnet_ids
private_subnet_ids = var.private_subnet_ids
create_database = var.create_database
database_url = var.database_url
create_redis = var.create_redis
redis_url = var.redis_url
litellm_master_key = var.litellm_master_key
litellm_license = var.litellm_license
ui_password = var.ui_password

27
terraform/litellm/aws/examples/default/terraform.tfvars.example
View file

@ -1,5 +1,30 @@
region = "us-west-2"
azs = ["us-west-2a", "us-west-2b"]
# Networking: by default the module creates a VPC, public/private subnets in
# each AZ listed here, an internet gateway, a NAT gateway, and route tables.
azs = ["us-west-2a", "us-west-2b"]
# To deploy into networking you already own, drop `azs` and set these
# instead. Nothing network-related is created then, so the private subnets
# need their own egress for LLM providers, image pulls, and Secrets Manager.
# vpc_id = "vpc-0123456789abcdef0"
# public_subnet_ids = ["subnet-aaa", "subnet-bbb"]
# private_subnet_ids = ["subnet-ccc", "subnet-ddd"]
# Data stores: Aurora Postgres and ElastiCache Redis are created by default.
# Set create_* = false to point at your own, passing a connection string
# (stored in Secrets Manager, injected as DATABASE_URL / REDIS_URL). Make
# sure they allow inbound from the stack's tasks security group, which the
# `task_security_group_id` output names.
# create_database = false
# database_url = "postgresql://litellm:...@db.internal:5432/litellm"
# create_redis = false
# redis_url = "rediss://:...@cache.internal:6379"
#
# Leaving the URL empty runs without that component: no database means no
# virtual keys, spend tracking, or UI persistence (master-key auth only), and
# no Redis means rate limits, budgets, and router cooldowns go per-task
# instead of cluster-wide.
# Resource naming: every AWS resource the stack creates is named
# `${tenant}-litellm-${env}` (or that plus a per-resource suffix). E.g.

52
terraform/litellm/aws/examples/default/variables.tf
View file

@ -21,8 +21,58 @@ variable "env" {
}
variable "azs" {
description = "Availability zones for subnets. At least 2 (RDS + ALB)."
description = "Availability zones for the subnets the module creates. At least 2 (RDS + ALB). Unused when vpc_id is set."
type = list(string)
default = []
}
# Bring-your-own networking. Leave vpc_id empty to have the module create the
# VPC, subnets, NAT gateway, and route tables.
variable "vpc_id" {
description = "Existing VPC to deploy into. Empty → module creates its own networking."
type = string
default = ""
}
variable "public_subnet_ids" {
description = "Existing public subnets for the ALB (≥ 2 AZs). Required with vpc_id."
type = list(string)
default = []
}
variable "private_subnet_ids" {
description = "Existing private subnets for tasks, Aurora, and Redis. Required with vpc_id."
type = list(string)
default = []
}
# Bring-your-own data stores. create_* false with an empty URL runs without
# that component: no DB means no key management or spend tracking, no Redis
# means per-task rate limits instead of cluster-wide.
variable "create_database" {
description = "Create the Aurora Postgres cluster. False → use database_url, or run DB-less."
type = bool
default = true
}
variable "database_url" {
description = "Postgres connection string for an existing database. Read only when create_database = false."
type = string
default = ""
sensitive = true
}
variable "create_redis" {
description = "Create the ElastiCache Redis group. False → use redis_url, or run without Redis."
type = bool
default = true
}
variable "redis_url" {
description = "Connection string for an existing Redis. Read only when create_redis = false."
type = string
default = ""
sensitive = true
}
# Sensitive — prefer TF_VAR_litellm_master_key / TF_VAR_litellm_license /

15
terraform/litellm/aws/iam.tf
View file

@ -56,6 +56,8 @@ data "aws_iam_policy_document" "secrets_access" {
aws_secretsmanager_secret.billing_metrics_client_cert[*].arn,
aws_secretsmanager_secret.billing_metrics_client_key[*].arn,
aws_secretsmanager_secret.billing_metrics_ca_cert[*].arn,
aws_secretsmanager_secret.database_url[*].arn,
aws_secretsmanager_secret.redis_url[*].arn,
local.extra_secret_arns,
var.otel_headers_secret_arn == "" ? [] : [var.otel_headers_secret_arn],
)
@ -79,6 +81,9 @@ resource "aws_iam_role_policy_attachment" "task_execution_secrets" {
# Assumed by the running container. Gets `rds-db:connect` so the proxy can
# mint IAM-signed Postgres tokens for the app user. Layer additional
# policies here (e.g. Bedrock invoke, S3 read) when the proxy needs them.
# IAM auth only applies to the Aurora cluster this module creates: an
# existing database is reached with the credentials embedded in
# var.database_url, so the policy is skipped there.
resource "aws_iam_role" "task" {
name = "${local.name}-task"
@ -90,24 +95,28 @@ resource "aws_iam_role" "task" {
data "aws_caller_identity" "current" {}
data "aws_iam_policy_document" "rds_iam_connect" {
count = var.create_database ? 1 : 0
statement {
actions = ["rds-db:connect"]
resources = [
"arn:aws:rds-db:${var.region}:${data.aws_caller_identity.current.account_id}:dbuser:${aws_rds_cluster.this.cluster_resource_id}/${var.db_username}",
"arn:aws:rds-db:${var.region}:${data.aws_caller_identity.current.account_id}:dbuser:${aws_rds_cluster.this[0].cluster_resource_id}/${var.db_username}",
]
}
}
resource "aws_iam_policy" "rds_iam_connect" {
count = var.create_database ? 1 : 0
name = "${local.name}-rds-iam-connect"
policy = data.aws_iam_policy_document.rds_iam_connect.json
policy = data.aws_iam_policy_document.rds_iam_connect[0].json
tags = local.tags
}
resource "aws_iam_role_policy_attachment" "task_rds_iam_connect" {
count = var.create_database ? 1 : 0
role = aws_iam_role.task.name
policy_arn = aws_iam_policy.rds_iam_connect.arn
policy_arn = aws_iam_policy.rds_iam_connect[0].arn
}
# ---------- UI task role ----------

21
terraform/litellm/aws/locals.tf
View file

@ -25,6 +25,27 @@ locals {
var.tags,
)
# Networking, database, and cache are each either module-owned or
# bring-your-own. Everything downstream reads these locals rather than the
# resources, so a resource going to zero instances doesn't ripple.
create_vpc = var.vpc_id == ""
vpc_id = local.create_vpc ? aws_vpc.this[0].id : var.vpc_id
public_subnet_ids = local.create_vpc ? aws_subnet.public[*].id : var.public_subnet_ids
private_subnet_ids = local.create_vpc ? aws_subnet.private[*].id : var.private_subnet_ids
task_security_group_ids = concat([aws_security_group.tasks.id], var.additional_task_security_group_ids)
# `byo_*` is the existing-store branch, `database_enabled` is either branch.
# Neither branch means the component is absent: no DB (no key management,
# spend tracking, or UI persistence) or no Redis (per-task rate limits and
# cooldowns instead of cluster-wide).
# nonsensitive() on the emptiness check only: without it the sensitivity of
# the URLs propagates into every value derived from these flags, redacting
# unrelated task-definition and output diffs in the plan.
byo_database = !var.create_database && nonsensitive(var.database_url != "")
byo_redis = !var.create_redis && nonsensitive(var.redis_url != "")
database_enabled = var.create_database || local.byo_database
gateway_path_prefixes = [
"/v1/chat/*", "/chat/*",
"/v1/completions*", "/completions*",

4
terraform/litellm/aws/migrations.tf
View file

@ -13,6 +13,7 @@
# every apply (after the IAM-authed user has been created). The
# `migration_run_command` output is preserved for break-glass manual re-runs.
resource "aws_ecs_task_definition" "migrations" {
count = local.database_enabled ? 1 : 0
family = "${local.name}-migrations"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
@ -32,11 +33,12 @@ resource "aws_ecs_task_definition" "migrations" {
# No entryPoint/command override — the image's ENTRYPOINT runs run.py.
environment = local.shared_env
secrets = local.byo_database_secrets
logConfiguration = {
logDriver = "awslogs"
options = {
awslogs-group = aws_cloudwatch_log_group.migrations.name
awslogs-group = aws_cloudwatch_log_group.migrations[0].name
awslogs-region = var.region
awslogs-stream-prefix = "migrations"
}

73
terraform/litellm/aws/network.tf
View file

@ -1,24 +1,34 @@
data "aws_availability_zones" "available" {
state = "available"
}
# Networking is created only when the caller didn't supply a VPC. With
# var.vpc_id set, every resource in this file except the security groups has
# zero instances and the stack consumes the caller's subnets through
# local.public_subnet_ids / local.private_subnet_ids (see locals.tf).
resource "aws_vpc" "this" {
count = local.create_vpc ? 1 : 0
cidr_block = var.vpc_cidr
enable_dns_hostnames = true
enable_dns_support = true
lifecycle {
precondition {
condition = length(var.azs) >= 2
error_message = "Provide at least 2 availability zones in `azs`, or set `vpc_id` + `public_subnet_ids` + `private_subnet_ids` to deploy into an existing VPC."
}
}
tags = merge(local.tags, { Name = local.name })
}
resource "aws_internet_gateway" "this" {
vpc_id = aws_vpc.this.id
count = local.create_vpc ? 1 : 0
vpc_id = aws_vpc.this[0].id
tags = merge(local.tags, { Name = local.name })
}
# Public subnets (ALB + NAT). One per AZ.
resource "aws_subnet" "public" {
count = length(var.azs)
vpc_id = aws_vpc.this.id
count = local.create_vpc ? length(var.azs) : 0
vpc_id = aws_vpc.this[0].id
cidr_block = cidrsubnet(var.vpc_cidr, 8, count.index)
availability_zone = var.azs[count.index]
map_public_ip_on_launch = true
@ -29,8 +39,8 @@ resource "aws_subnet" "public" {
# Private subnets (ECS tasks, RDS, ElastiCache). One per AZ, separate from
# public range.
resource "aws_subnet" "private" {
count = length(var.azs)
vpc_id = aws_vpc.this.id
count = local.create_vpc ? length(var.azs) : 0
vpc_id = aws_vpc.this[0].id
cidr_block = cidrsubnet(var.vpc_cidr, 8, count.index + 10)
availability_zone = var.azs[count.index]
@ -38,6 +48,7 @@ resource "aws_subnet" "private" {
}
resource "aws_eip" "nat" {
count = local.create_vpc ? 1 : 0
domain = "vpc"
tags = merge(local.tags, { Name = "${local.name}-nat" })
@ -47,7 +58,8 @@ resource "aws_eip" "nat" {
# Single NAT gateway in the first public subnet. For HA, replicate per AZ —
# adds ~$30/mo per gateway, so off by default for a baseline deployment.
resource "aws_nat_gateway" "this" {
allocation_id = aws_eip.nat.id
count = local.create_vpc ? 1 : 0
allocation_id = aws_eip.nat[0].id
subnet_id = aws_subnet.public[0].id
tags = merge(local.tags, { Name = local.name })
@ -56,45 +68,53 @@ resource "aws_nat_gateway" "this" {
}
resource "aws_route_table" "public" {
vpc_id = aws_vpc.this.id
count = local.create_vpc ? 1 : 0
vpc_id = aws_vpc.this[0].id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.this.id
gateway_id = aws_internet_gateway.this[0].id
}
tags = merge(local.tags, { Name = "${local.name}-public" })
}
resource "aws_route_table_association" "public" {
count = length(var.azs)
count = local.create_vpc ? length(var.azs) : 0
subnet_id = aws_subnet.public[count.index].id
route_table_id = aws_route_table.public.id
route_table_id = aws_route_table.public[0].id
}
resource "aws_route_table" "private" {
vpc_id = aws_vpc.this.id
count = local.create_vpc ? 1 : 0
vpc_id = aws_vpc.this[0].id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.this.id
nat_gateway_id = aws_nat_gateway.this[0].id
}
tags = merge(local.tags, { Name = "${local.name}-private" })
}
resource "aws_route_table_association" "private" {
count = length(var.azs)
count = local.create_vpc ? length(var.azs) : 0
subnet_id = aws_subnet.private[count.index].id
route_table_id = aws_route_table.private.id
route_table_id = aws_route_table.private[0].id
}
# ---------- Security groups ----------
#
# Always module-owned, in local.vpc_id, so the stack keeps a least-privilege
# path between its own components even when it borrows someone else's VPC.
# Existing databases and caches reached over var.database_url / var.redis_url
# need to allow inbound from the tasks group (or from a group passed via
# var.additional_task_security_group_ids).
resource "aws_security_group" "alb" {
name = "${local.name}-alb"
description = "Inbound HTTP/HTTPS to the LiteLLM ALB."
vpc_id = aws_vpc.this.id
vpc_id = local.vpc_id
ingress {
description = "HTTP from anywhere"
@ -126,7 +146,7 @@ resource "aws_security_group" "alb" {
resource "aws_security_group" "tasks" {
name = "${local.name}-tasks"
description = "ECS tasks (gateway/backend/ui)."
vpc_id = aws_vpc.this.id
vpc_id = local.vpc_id
ingress {
description = "ALB to tasks"
@ -144,13 +164,23 @@ resource "aws_security_group" "tasks" {
cidr_blocks = ["0.0.0.0/0"]
}
# The tasks group is created in every mode, so this is where the
# bring-your-own-VPC inputs get checked.
lifecycle {
precondition {
condition = local.create_vpc || length(var.private_subnet_ids) > 0
error_message = "`private_subnet_ids` is required when `vpc_id` is set: the tasks, Aurora, and ElastiCache all live in private subnets."
}
}
tags = local.tags
}
resource "aws_security_group" "rds" {
count = var.create_database ? 1 : 0
name = "${local.name}-rds"
description = "RDS Postgres - tasks only."
vpc_id = aws_vpc.this.id
vpc_id = local.vpc_id
ingress {
description = "Postgres from ECS tasks"
@ -164,9 +194,10 @@ resource "aws_security_group" "rds" {
}
resource "aws_security_group" "redis" {
count = var.create_redis ? 1 : 0
name = "${local.name}-redis"
description = "ElastiCache Redis - tasks only."
vpc_id = aws_vpc.this.id
vpc_id = local.vpc_id
ingress {
description = "Redis from ECS tasks"

42
terraform/litellm/aws/outputs.tf
View file

@ -13,19 +13,29 @@ output "ecs_cluster" {
value = aws_ecs_cluster.this.name
}
output "vpc_id" {
description = "VPC the stack runs in, whether module-created or passed in via `vpc_id`."
value = local.vpc_id
}
output "task_security_group_id" {
description = "Security group attached to the ECS tasks. Allow inbound from this group on an existing database or Redis reached over `database_url` / `redis_url`."
value = aws_security_group.tasks.id
}
output "aurora_writer_endpoint" {
description = "Aurora writer endpoint (cluster endpoint). Used by gateway/backend as DATABASE_HOST."
value = aws_rds_cluster.this.endpoint
description = "Aurora writer endpoint (cluster endpoint). Used by gateway/backend as DATABASE_HOST. Null when `create_database = false`."
value = one(aws_rds_cluster.this[*].endpoint)
}
output "aurora_reader_endpoint" {
description = "Aurora reader endpoint. Used by gateway/backend as DATABASE_HOST_READ_REPLICA."
value = aws_rds_cluster.this.reader_endpoint
description = "Aurora reader endpoint. Used by gateway/backend as DATABASE_HOST_READ_REPLICA. Null when `create_database = false`."
value = one(aws_rds_cluster.this[*].reader_endpoint)
}
output "redis_endpoint" {
description = "ElastiCache Redis primary endpoint (TLS, transit_encryption_enabled = true)."
value = "${aws_elasticache_replication_group.this.primary_endpoint_address}:${aws_elasticache_replication_group.this.port}"
description = "ElastiCache Redis primary endpoint (TLS, transit_encryption_enabled = true). Null when `create_redis = false`."
value = one([for r in aws_elasticache_replication_group.this : "${r.primary_endpoint_address}:${r.port}"])
}
output "s3_bucket" {
@ -39,15 +49,17 @@ output "master_key_secret_arn" {
}
output "db_master_password_secret_arn" {
description = "Secrets Manager ARN holding the Aurora master credentials (bootstrap-only). Used to create the IAM-authed application user."
value = aws_secretsmanager_secret.db_master_password.arn
description = "Secrets Manager ARN holding the Aurora master credentials (bootstrap-only). Used to create the IAM-authed application user. Null when `create_database = false`."
value = one(aws_secretsmanager_secret.db_master_password[*].arn)
}
# Pre-baked SQL to run once as the master user, creating the IAM-authed
# application user that gateway/backend/migration tasks will authenticate as.
# Irrelevant to an existing database reached over `database_url`, whose
# credentials are already in the URL.
output "db_bootstrap_sql" {
description = "Run this once as the master DB user (after the first apply) to create the IAM-authed app user."
value = <<-SQL
description = "Run this once as the master DB user (after the first apply) to create the IAM-authed app user. Empty when `create_database = false`."
value = !var.create_database ? "" : <<-SQL
CREATE USER ${var.db_username};
GRANT rds_iam TO ${var.db_username};
GRANT ALL PRIVILEGES ON DATABASE ${var.db_name} TO ${var.db_username};
@ -60,13 +72,13 @@ output "db_bootstrap_sql" {
# Pre-baked command for running the one-off migration task. ECS run-task
# needs the subnet + SG IDs at call time, so we render the full command.
output "migration_run_command" {
description = "Shell command that runs the one-off prisma migration task against Aurora. Run this once, after the bootstrap SQL above, before sending traffic."
value = format(
description = "Shell command that runs the one-off prisma migration task against the database. Run this once, after the bootstrap SQL above, before sending traffic. Empty when the stack has no database."
value = !local.database_enabled ? "" : format(
"aws ecs run-task --cluster %s --launch-type FARGATE --task-definition %s --network-configuration 'awsvpcConfiguration={subnets=[%s],securityGroups=[%s],assignPublicIp=DISABLED}' --region %s",
aws_ecs_cluster.this.name,
aws_ecs_task_definition.migrations.arn,
join(",", aws_subnet.private[*].id),
aws_security_group.tasks.id,
aws_ecs_task_definition.migrations[0].arn,
join(",", local.private_subnet_ids),
join(",", local.task_security_group_ids),
var.region,
)
}

31
terraform/litellm/aws/rds.tf
View file

@ -1,5 +1,7 @@
# Aurora Postgres cluster with one writer + one reader instance, IAM
# database authentication enabled.
# database authentication enabled. Skipped entirely when
# create_database = false, in which case the stack either talks to the
# database named by var.database_url or runs without one.
#
# Important: enabling IAM auth on the cluster does not by itself grant any
# Postgres user the ability to log in with an IAM token. After the first
@ -17,13 +19,15 @@
# superusers — keep it for break-glass only.
resource "aws_db_subnet_group" "this" {
count = var.create_database ? 1 : 0
name = "${local.name}-db"
subnet_ids = aws_subnet.private[*].id
subnet_ids = local.private_subnet_ids
tags = local.tags
}
resource "aws_rds_cluster_parameter_group" "this" {
count = var.create_database ? 1 : 0
name = "${local.name}-cluster-pg"
family = "aurora-postgresql${split(".", var.db_engine_version)[0]}"
description = "LiteLLM Aurora Postgres cluster parameters."
@ -32,16 +36,17 @@ resource "aws_rds_cluster_parameter_group" "this" {
}
resource "aws_rds_cluster" "this" {
count = var.create_database ? 1 : 0
cluster_identifier = local.name
engine = "aurora-postgresql"
engine_mode = "provisioned"
engine_version = var.db_engine_version
database_name = var.db_name
master_username = var.db_master_username
master_password = random_password.db_master_password.result
db_subnet_group_name = aws_db_subnet_group.this.name
vpc_security_group_ids = [aws_security_group.rds.id]
db_cluster_parameter_group_name = aws_rds_cluster_parameter_group.this.name
master_password = random_password.db_master_password[0].result
db_subnet_group_name = aws_db_subnet_group.this[0].name
vpc_security_group_ids = [aws_security_group.rds[0].id]
db_cluster_parameter_group_name = aws_rds_cluster_parameter_group.this[0].name
iam_database_authentication_enabled = true
storage_encrypted = true
@ -61,11 +66,12 @@ resource "aws_rds_cluster" "this" {
}
resource "aws_rds_cluster_instance" "writer" {
count = var.create_database ? 1 : 0
identifier = "${local.name}-writer"
cluster_identifier = aws_rds_cluster.this.id
cluster_identifier = aws_rds_cluster.this[0].id
instance_class = var.db_instance_class
engine = aws_rds_cluster.this.engine
engine_version = aws_rds_cluster.this.engine_version
engine = aws_rds_cluster.this[0].engine
engine_version = aws_rds_cluster.this[0].engine_version
publicly_accessible = false
performance_insights_enabled = true
@ -78,11 +84,12 @@ resource "aws_rds_cluster_instance" "writer" {
}
resource "aws_rds_cluster_instance" "reader" {
count = var.create_database ? 1 : 0
identifier = "${local.name}-reader"
cluster_identifier = aws_rds_cluster.this.id
cluster_identifier = aws_rds_cluster.this[0].id
instance_class = var.db_instance_class
engine = aws_rds_cluster.this.engine
engine_version = aws_rds_cluster.this.engine_version
engine = aws_rds_cluster.this[0].engine
engine_version = aws_rds_cluster.this[0].engine_version
publicly_accessible = false
performance_insights_enabled = true

8
terraform/litellm/aws/redis.tf
View file

@ -1,6 +1,7 @@
resource "aws_elasticache_subnet_group" "this" {
count = var.create_redis ? 1 : 0
name = "${local.name}-redis"
subnet_ids = aws_subnet.private[*].id
subnet_ids = local.private_subnet_ids
tags = local.tags
}
@ -13,6 +14,7 @@ resource "aws_elasticache_subnet_group" "this" {
# TLS-protected — the proxy connects via the rediss:// scheme thanks to
# REDIS_SSL=true in the shared task env (see ecs.tf).
resource "aws_elasticache_replication_group" "this" {
count = var.create_redis ? 1 : 0
replication_group_id = "${local.name}-redis"
description = "LiteLLM ElastiCache Redis"
@ -23,8 +25,8 @@ resource "aws_elasticache_replication_group" "this" {
parameter_group_name = "default.redis7"
port = 6379
subnet_group_name = aws_elasticache_subnet_group.this.name
security_group_ids = [aws_security_group.redis.id]
subnet_group_name = aws_elasticache_subnet_group.this[0].name
security_group_ids = [aws_security_group.redis[0].id]
automatic_failover_enabled = var.redis_num_replicas >= 1
multi_az_enabled = var.redis_num_replicas >= 1

48
terraform/litellm/aws/secrets.tf
View file

@ -10,6 +10,7 @@ resource "random_password" "master_key" {
# user (see rds.tf header). Runtime services authenticate via IAM tokens
# and never read this secret.
resource "random_password" "db_master_password" {
count = var.create_database ? 1 : 0
length = 32
special = false
min_lower = 4
@ -130,6 +131,7 @@ resource "aws_secretsmanager_secret_version" "billing_metrics_ca_cert" {
}
resource "aws_secretsmanager_secret" "db_master_password" {
count = var.create_database ? 1 : 0
name = "${local.name}-db-master-password"
description = "Aurora master-user password - bootstrap only. Runtime auth is IAM-token."
recovery_window_in_days = 0
@ -138,12 +140,50 @@ resource "aws_secretsmanager_secret" "db_master_password" {
}
resource "aws_secretsmanager_secret_version" "db_master_password" {
secret_id = aws_secretsmanager_secret.db_master_password.id
count = var.create_database ? 1 : 0
secret_id = aws_secretsmanager_secret.db_master_password[0].id
secret_string = jsonencode({
username = var.db_master_username
password = random_password.db_master_password.result
host = aws_rds_cluster.this.endpoint
port = aws_rds_cluster.this.port
password = random_password.db_master_password[0].result
host = aws_rds_cluster.this[0].endpoint
port = aws_rds_cluster.this[0].port
dbname = var.db_name
})
}
# Bring-your-own connection strings. Both hold credentials, so they go to
# Secrets Manager and reach the containers as ECS `secrets` rather than as
# plain-text env in the task definition.
resource "aws_secretsmanager_secret" "database_url" {
count = local.byo_database ? 1 : 0
name = "${local.name}-database-url"
description = "DATABASE_URL for an existing Postgres, used when create_database = false."
recovery_window_in_days = 0
tags = local.tags
}
resource "aws_secretsmanager_secret_version" "database_url" {
count = local.byo_database ? 1 : 0
secret_id = aws_secretsmanager_secret.database_url[0].id
secret_string = var.database_url
}
resource "aws_secretsmanager_secret" "redis_url" {
count = local.byo_redis ? 1 : 0
name = "${local.name}-redis-url"
description = "REDIS_URL for an existing Redis, used when create_redis = false."
recovery_window_in_days = 0
tags = local.tags
}
resource "aws_secretsmanager_secret_version" "redis_url" {
count = local.byo_redis ? 1 : 0
secret_id = aws_secretsmanager_secret.redis_url[0].id
secret_string = var.redis_url
}

176
terraform/litellm/aws/tests/byo_infrastructure.tftest.hcl Normal file
View file

@ -0,0 +1,176 @@
# Plan-only coverage for the four networking/database/cache permutations.
# `mock_provider` keeps this offline: no AWS credentials, no API calls, no
# resources. Run from terraform/litellm/aws with `terraform test`.
mock_provider "aws" {
# IAM policy documents are validated as JSON by the provider, so the
# generated placeholder string has to be replaced with a parsable one.
mock_data "aws_iam_policy_document" {
defaults = {
json = "{\"Version\":\"2012-10-17\",\"Statement\":[]}"
}
}
}
mock_provider "random" {}
variables {
region = "us-east-1"
tenant = "acme"
env = "test"
allow_plaintext_alb = true
}
run "module_owns_everything_by_default" {
command = plan
variables {
azs = ["us-east-1a", "us-east-1b"]
}
assert {
condition = length(aws_vpc.this) == 1 && length(aws_nat_gateway.this) == 1 && length(aws_subnet.private) == 2
error_message = "The default path must still create its own VPC, NAT gateway, and one private subnet per AZ."
}
assert {
condition = length(aws_rds_cluster.this) == 1 && length(aws_elasticache_replication_group.this) == 1
error_message = "The default path must still create Aurora and ElastiCache."
}
assert {
condition = length(aws_secretsmanager_secret.database_url) == 0 && length(aws_secretsmanager_secret.redis_url) == 0
error_message = "Connection-string secrets belong to the bring-your-own path only."
}
assert {
condition = length(local.managed_db_env) == 7 && length(local.managed_redis_env) == 3
error_message = "Gateway, backend, and migration tasks must keep the discrete DATABASE_*/REDIS_* env for the module-created stores."
}
assert {
condition = length(terraform_data.bootstrap_db) == 1 && length(aws_ecs_task_definition.migrations) == 1
error_message = "The IAM-user bootstrap and the schema migration must both run against the module-created Aurora."
}
}
run "existing_vpc_creates_no_networking" {
command = plan
variables {
vpc_id = "vpc-00000000000000001"
public_subnet_ids = ["subnet-pub-a", "subnet-pub-b"]
private_subnet_ids = ["subnet-priv-a", "subnet-priv-b"]
additional_task_security_group_ids = ["sg-caller-owned"]
}
assert {
condition = alltrue([
length(aws_vpc.this) == 0,
length(aws_subnet.public) == 0,
length(aws_subnet.private) == 0,
length(aws_internet_gateway.this) == 0,
length(aws_nat_gateway.this) == 0,
length(aws_eip.nat) == 0,
length(aws_route_table.public) == 0,
length(aws_route_table.private) == 0,
])
error_message = "An existing vpc_id must suppress every network resource, including the route tables and NAT gateway."
}
assert {
condition = aws_lb.this.subnets == toset(var.public_subnet_ids)
error_message = "The ALB must land in the caller's public subnets."
}
assert {
condition = alltrue([
aws_db_subnet_group.this[0].subnet_ids == toset(var.private_subnet_ids),
aws_elasticache_subnet_group.this[0].subnet_ids == toset(var.private_subnet_ids),
aws_ecs_service.gateway.network_configuration[0].subnets == toset(var.private_subnet_ids),
])
error_message = "Tasks, Aurora, and ElastiCache must land in the caller's private subnets."
}
assert {
condition = length(local.task_security_group_ids) == 2
error_message = "additional_task_security_group_ids must be attached alongside the module's own tasks group."
}
}
run "existing_database_and_redis_replace_the_managed_ones" {
command = plan
variables {
azs = ["us-east-1a", "us-east-1b"]
create_database = false
database_url = "postgresql://litellm:pw@db.internal:5432/litellm"
create_redis = false
redis_url = "rediss://:pw@cache.internal:6379"
}
assert {
condition = alltrue([
length(aws_rds_cluster.this) == 0,
length(aws_rds_cluster_instance.writer) == 0,
length(aws_db_subnet_group.this) == 0,
length(aws_security_group.rds) == 0,
length(aws_elasticache_replication_group.this) == 0,
length(aws_elasticache_subnet_group.this) == 0,
length(aws_security_group.redis) == 0,
])
error_message = "Pointing at an existing database and cache must create neither Aurora nor ElastiCache."
}
assert {
condition = length(local.managed_db_env) == 0 && length(local.managed_redis_env) == 0
error_message = "The discrete DATABASE_*/REDIS_* env vars must be dropped so DATABASE_URL/REDIS_URL are the only connection targets."
}
assert {
condition = alltrue([
length([for s in local.shared_secrets : s if s.name == "DATABASE_URL"]) == 1,
length([for s in local.shared_secrets : s if s.name == "REDIS_URL"]) == 1,
])
error_message = "Both connection strings must reach the containers as Secrets Manager references, not plain-text env."
}
assert {
condition = length(terraform_data.bootstrap_db) == 0 && length(aws_ecs_task_definition.migrations) == 1
error_message = "An existing database still needs the schema migration, but not the Aurora IAM-user bootstrap."
}
assert {
condition = length([for e in local.backend_default_env : e if e.name == "STORE_MODEL_IN_DB"]) == 1
error_message = "STORE_MODEL_IN_DB must stay set when a database is reachable."
}
}
run "no_database_and_no_redis_drops_the_schema_migration" {
command = plan
variables {
azs = ["us-east-1a", "us-east-1b"]
create_database = false
create_redis = false
}
assert {
condition = alltrue([
length(aws_ecs_task_definition.migrations) == 0,
length(terraform_data.migration) == 0,
length(aws_iam_policy.rds_iam_connect) == 0,
length(aws_secretsmanager_secret.db_master_password) == 0,
])
error_message = "With no database at all there is nothing to migrate, bootstrap, or grant rds-db:connect on."
}
assert {
condition = length(local.backend_default_env) == 0
error_message = "STORE_MODEL_IN_DB must not be set without a database to store models in."
}
assert {
condition = length(local.shared_env) == 4
error_message = "The shared env must narrow to the S3 bucket and region pair when both data stores are gone."
}
}

108
terraform/litellm/aws/variables.tf
View file

@ -74,20 +74,63 @@ variable "ui_password" {
}
# ---------- Networking ----------
#
# Two modes:
#
# 1. Module-owned (default, `vpc_id = ""`): the stack creates a VPC, public
# and private subnets per AZ, an internet gateway, a NAT gateway, and
# the route tables wiring them together. `vpc_cidr` + `azs` drive it.
# 2. Bring-your-own (`vpc_id` set): the stack creates no networking and
# places the ALB in `public_subnet_ids` and every task, plus the Aurora
# and ElastiCache subnet groups, in `private_subnet_ids`. `vpc_cidr` and
# `azs` are then unused.
variable "vpc_id" {
description = <<-EOT
Existing VPC to deploy into. Leave empty ("") to have the module create
its own VPC, subnets, NAT gateway, and route tables. When set,
`public_subnet_ids` and `private_subnet_ids` are required and no
networking is created: the private subnets must already have egress
(NAT gateway or equivalent) so tasks can reach LLM providers, ECR/GHCR,
and Secrets Manager.
EOT
type = string
default = ""
}
variable "public_subnet_ids" {
description = "Existing public subnets for the ALB, in at least 2 AZs. Required when `vpc_id` is set, ignored otherwise."
type = list(string)
default = []
}
variable "private_subnet_ids" {
description = "Existing private subnets for the ECS tasks, Aurora, and ElastiCache. Required when `vpc_id` is set, ignored otherwise."
type = list(string)
default = []
}
variable "additional_task_security_group_ids" {
description = <<-EOT
Extra security groups to attach to the ECS tasks, on top of the one the
module creates. Useful with `vpc_id`: attach a group your existing
database or cache already allows inbound from, instead of editing their
ingress rules.
EOT
type = list(string)
default = []
}
variable "vpc_cidr" {
description = "CIDR block for the VPC."
description = "CIDR block for the VPC the module creates. Unused when `vpc_id` is set."
type = string
default = "10.40.0.0/16"
}
variable "azs" {
description = "Availability zones to spread subnets across. At least 2 required for RDS and ALB."
description = "Availability zones to spread the module-created subnets across. At least 2 required for Aurora and the ALB. Unused when `vpc_id` is set."
type = list(string)
validation {
condition = length(var.azs) >= 2
error_message = "Provide at least 2 availability zones."
}
default = []
}
# ---------- Component images ----------
@ -279,6 +322,34 @@ variable "ui_cpu_target" {
# ---------- RDS ----------
variable "create_database" {
description = <<-EOT
Create the Aurora Postgres cluster (default). Set false to skip it and
either point the stack at an existing database via `database_url`, or
run without a database at all when `database_url` is also empty. The
DB-less mode drops key management, spend tracking, and the admin UI's
persistence: the proxy then serves traffic authenticated by
LITELLM_MASTER_KEY only.
EOT
type = bool
default = true
}
variable "database_url" {
description = <<-EOT
Postgres connection string for an existing database, e.g.
`postgresql://user:pass@host:5432/litellm`. Only read when
`create_database = false`. Stored in a
`<tenant>-litellm-<env>-database-url` Secrets Manager entry and injected
into gateway, backend, and the migration task as DATABASE_URL, so the
value never lands in a task definition. The schema migration still runs
against it on every apply.
EOT
type = string
default = ""
sensitive = true
}
variable "db_instance_class" {
description = "Aurora instance class for both writer and reader."
type = string
@ -311,6 +382,31 @@ variable "db_username" {
# ---------- Redis ----------
variable "create_redis" {
description = <<-EOT
Create the ElastiCache Redis replication group (default). Set false to
skip it and either point the stack at an existing cache via `redis_url`,
or run with no Redis at all when `redis_url` is also empty. Without
Redis the proxy loses cross-task state: rate limits, budgets, and the
router's cooldowns become per-task instead of cluster-wide.
EOT
type = bool
default = true
}
variable "redis_url" {
description = <<-EOT
Connection string for an existing Redis, e.g.
`rediss://:password@host:6379`. Only read when `create_redis = false`.
Stored in a `<tenant>-litellm-<env>-redis-url` Secrets Manager entry and
injected as REDIS_URL, which takes precedence over REDIS_HOST/REDIS_PORT
in the proxy.
EOT
type = string
default = ""
sensitive = true
}
variable "redis_node_type" {
description = "ElastiCache node type."
type = string