From ccd8c4112c2092d680fcfc60a2b20da1b722d5e9 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 17:38:48 -0700 Subject: [PATCH 01/12] docs(readme): add Deploy on AWS/GCP with Terraform section Adds a quickstart for the two published Terraform modules on the public registry (BerriAI/litellm/aws and BerriAI/litellm/google). Copy-paste main.tf for each cloud, the one-time GCP Artifact Registry remote-repo command, and pointers to the registry pages for the full input surface. Sits inside the Get Started section, between the gateway/SDK table and Run in Developer Mode -- where someone scanning the README for "how do I deploy this" will land. Co-Authored-By: Claude Opus 4.7 --- README.md | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) diff --git a/README.md b/README.md index d600f3952c6..faf79c3729d 100644 --- a/README.md +++ b/README.md @@ -404,6 +404,124 @@ You can use LiteLLM through either the Proxy Server or Python SDK. Both give you Support for more providers. Missing a provider or LLM Platform, raise a [feature request](https://github.com/BerriAI/litellm/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.yml&title=%5BFeature%5D%3A+). +### Deploy on AWS or GCP with Terraform + +Run the LiteLLM proxy as a production-ready componentized stack (gateway, backend, UI on separate services; managed Postgres + Redis + object store) using the published Terraform modules. Both modules are on the [public Terraform Registry](https://registry.terraform.io/namespaces/BerriAI) — no auth needed. + +#### AWS — ECS Fargate + Aurora + ElastiCache + ALB + +[Module page →](https://registry.terraform.io/modules/BerriAI/litellm/aws/latest) + +```hcl +# main.tf +terraform { + required_version = ">= 1.6.0" + required_providers { + aws = { source = "hashicorp/aws", version = "~> 5.60" } + } +} + +provider "aws" { + region = "us-west-2" +} + +module "litellm" { + source = "BerriAI/litellm/aws" + version = "~> 1.89" + + region = "us-west-2" + azs = ["us-west-2a", "us-west-2b"] + tenant = "acme" + env = "prod" + + # Production: provide an ACM cert. Without one, set allow_plaintext_alb = true + # (dev/trial only). + # acm_certificate_arn = "arn:aws:acm:us-west-2:111122223333:certificate/..." + allow_plaintext_alb = true +} + +output "litellm_url" { + value = module.litellm.alb_dns_name +} +``` + +```bash +terraform init +terraform apply +``` + +Provider API keys live in AWS Secrets Manager; reference ARNs via `gateway_extra_secrets`. Full input list and architecture diagram on the [registry page](https://registry.terraform.io/modules/BerriAI/litellm/aws/latest?tab=inputs). + +#### GCP — Cloud Run + Cloud SQL + Memorystore + HTTPS LB + +[Module page →](https://registry.terraform.io/modules/BerriAI/litellm/google/latest) + +Cloud Run can't pull from `ghcr.io` directly, so first set up a one-time Artifact Registry remote repo backed by GHCR: + +```bash +gcloud artifacts repositories create litellm \ + --location=us-central1 \ + --repository-format=docker \ + --mode=remote-repository \ + --remote-docker-repo=https://ghcr.io \ + --project=my-gcp-project +``` + +Then: + +```hcl +# main.tf +terraform { + required_version = ">= 1.6.0" + required_providers { + google = { source = "hashicorp/google", version = "~> 6.10" } + google-beta = { source = "hashicorp/google-beta", version = "~> 6.10" } + } +} + +provider "google" { project = "my-gcp-project"; region = "us-central1" } +provider "google-beta" { project = "my-gcp-project"; region = "us-central1" } + +module "litellm" { + source = "BerriAI/litellm/google" + version = "~> 1.89" + + project_id = "my-gcp-project" + region = "us-central1" + tenant = "acme" + env = "prod" + + image_registry = "us-central1-docker.pkg.dev/my-gcp-project/litellm/berriai" + + # Production: provide DNS already pointing at the LB IP for Google-managed certs. + # Without one, set allow_plaintext_lb = true (dev/trial only). + # lb_domains = ["proxy.example.com"] + allow_plaintext_lb = true +} + +output "litellm_url" { + value = module.litellm.load_balancer_url +} +``` + +```bash +terraform init +terraform apply +``` + +Provider API keys live in Secret Manager; reference resource IDs (e.g. `projects/my-gcp-project/secrets/openai-api-key`) via `gateway_extra_secrets`. Full input list and architecture diagram on the [registry page](https://registry.terraform.io/modules/BerriAI/litellm/google/latest?tab=inputs). + +#### Both stacks include + +- The full componentized split (gateway / backend / UI as independent services) +- Managed Postgres (writer + reader) and Redis +- Versioned object store for proxy state + file uploads +- An auto-generated `LITELLM_MASTER_KEY` in your cloud's secret manager +- A one-off migration job that runs `prisma migrate deploy` before the proxy starts +- The same `proxy_config` surface as the [Helm chart](./helm/litellm/) — pass YAML as a typed map + +The Terraform modules live at [`terraform/litellm/aws/`](./terraform/litellm/aws/) and [`terraform/litellm/gcp/`](./terraform/litellm/gcp/) in this repo; the registry entries are read-only mirrors updated on each release. + ### Run in Developer Mode #### Services 1. Setup .env file in root From 260f5f1371e3308810cdaab61daa4da231cc1e76 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 17:44:49 -0700 Subject: [PATCH 02/12] docs(readme): add 1-click deploy buttons for AWS + GCP GCP gets the real 1-click: Open in Cloud Shell badge that clones the repo and walks through `terraform apply` via the existing DeployStack tutorial (already shipped at terraform/litellm/gcp/examples/default/ TUTORIAL.md). User just picks a project. AWS gets a soft 1-click: a Launch in AWS CloudShell badge that opens an in-browser, already-authenticated shell. User runs four commands (clone + cd + cp tfvars + terraform apply) once inside. There's no native AWS deeplink that pre-clones a repo + runs a tutorial -- CFN "Launch Stack" + CodeBuild would be needed for that, and that's a separate piece of work. Co-Authored-By: Claude Opus 4.7 --- README.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index faf79c3729d..d304d15f169 100644 --- a/README.md +++ b/README.md @@ -410,8 +410,19 @@ Run the LiteLLM proxy as a production-ready componentized stack (gateway, backen #### AWS — ECS Fargate + Aurora + ElastiCache + ALB +[![Launch in AWS CloudShell](https://img.shields.io/badge/Launch-AWS_CloudShell-FF9900?logo=amazon-aws&logoColor=white)](https://console.aws.amazon.com/cloudshell/home) — opens an in-browser shell, already authenticated to your AWS account. Once inside, run: + +```bash +git clone https://github.com/BerriAI/litellm.git +cd litellm/terraform/litellm/aws/examples/default +cp terraform.tfvars.example terraform.tfvars # edit region/tenant/env +terraform init && terraform apply +``` + [Module page →](https://registry.terraform.io/modules/BerriAI/litellm/aws/latest) +Or call the module from your own root config: + ```hcl # main.tf terraform { @@ -454,9 +465,13 @@ Provider API keys live in AWS Secrets Manager; reference ARNs via `gateway_extra #### GCP — Cloud Run + Cloud SQL + Memorystore + HTTPS LB +[![Open in Cloud Shell](https://gstatic.com/cloudssh/images/open-btn.svg)](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FBerriAI%2Flitellm&cloudshell_workspace=terraform%2Flitellm%2Fgcp%2Fexamples%2Fdefault&cloudshell_tutorial=TUTORIAL.md&cloudshell_image=gcr.io/ds-artifacts-cloudshell/deploystack_custom_image&shellonly=true) + +Real 1-click. Opens Cloud Shell, clones this repo, and walks you through `terraform apply` via a built-in [DeployStack tutorial](./terraform/litellm/gcp/examples/default/TUTORIAL.md) — pick the project, the tutorial sets up the Artifact Registry remote repo, writes `terraform.tfvars` from your answers, and runs apply. + [Module page →](https://registry.terraform.io/modules/BerriAI/litellm/google/latest) -Cloud Run can't pull from `ghcr.io` directly, so first set up a one-time Artifact Registry remote repo backed by GHCR: +To call the module from your own config instead, Cloud Run can't pull from `ghcr.io` directly, so first set up a one-time Artifact Registry remote repo backed by GHCR: ```bash gcloud artifacts repositories create litellm \ From a505734a2cddcece2d27ae7dea2e192f0f8b7a13 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 17:50:53 -0700 Subject: [PATCH 03/12] docs(readme): move AWS + GCP deploy buttons next to Render button --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index d304d15f169..7802e33baee 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,8 @@ Deploy on Railway + Launch in AWS CloudShell + Open in Cloud Shell

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

From d0ff1753973ee16b2c9ef88d34d2647e3d7cbec4 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 17:54:38 -0700 Subject: [PATCH 04/12] docs(readme): unify deploy button sizes and badge styles --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 7802e33baee..2509b761563 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,12 @@

Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.

- Deploy to Render + Deploy to Render - Deploy on Railway + Deploy on Railway - Launch in AWS CloudShell - Open in Cloud Shell + Deploy on AWS + Deploy on GCP

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

From daba177a5205f1be70a658a5dd063b23fd67e21f Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 18:15:36 -0700 Subject: [PATCH 05/12] docs(readme): bump deploy button height to 48 to match Render/Railway --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 2509b761563..693aee14683 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,12 @@

Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.

- Deploy to Render + Deploy to Render - Deploy on Railway + Deploy on Railway - Deploy on AWS - Deploy on GCP + Deploy on AWS + Deploy on GCP

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

From 1a18184f0eae4b331946ae564fd51dc7dbf1a0a2 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 18:25:49 -0700 Subject: [PATCH 06/12] docs(readme): bump AWS/GCP badge height to compensate for SVG padding --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 693aee14683..d97f07a1b8b 100644 --- a/README.md +++ b/README.md @@ -10,8 +10,8 @@ Deploy on Railway - Deploy on AWS - Deploy on GCP + Deploy on AWS + Deploy on GCP

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

From 8ba452c0ecfaf6b81268eb596ba2fb0d463bee06 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 18:26:21 -0700 Subject: [PATCH 07/12] docs(readme): bump AWS/GCP badge height to 72 --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d97f07a1b8b..91246629ad6 100644 --- a/README.md +++ b/README.md @@ -10,8 +10,8 @@ Deploy on Railway - Deploy on AWS - Deploy on GCP + Deploy on AWS + Deploy on GCP

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

From 2c56b4d3f64ae941322b78b35df4bf0dfea51f34 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 18:27:10 -0700 Subject: [PATCH 08/12] docs(readme): bump AWS/GCP badge height to 84 --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 91246629ad6..07a74296fd1 100644 --- a/README.md +++ b/README.md @@ -10,8 +10,8 @@ Deploy on Railway - Deploy on AWS - Deploy on GCP + Deploy on AWS + Deploy on GCP

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

From 65890b732e1e67af0d3c09a318197534d8966ced Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 7 Jun 2026 01:33:43 +0000 Subject: [PATCH 09/12] fix(readme): make deploy buttons same height (48px) https://claude.ai/code/session_01MxQRMHSDXbqJh74rF86UBc --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 07a74296fd1..693aee14683 100644 --- a/README.md +++ b/README.md @@ -10,8 +10,8 @@ Deploy on Railway - Deploy on AWS - Deploy on GCP + Deploy on AWS + Deploy on GCP

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

From 97f067bc70d7006f7aeb8dc117412c4d0f67e514 Mon Sep 17 00:00:00 2001 From: Yassin Kortam Date: Sat, 6 Jun 2026 18:37:28 -0700 Subject: [PATCH 10/12] docs(readme): flag GCP project ID substitution in image_registry --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 693aee14683..529e1de3354 100644 --- a/README.md +++ b/README.md @@ -508,6 +508,7 @@ module "litellm" { tenant = "acme" env = "prod" + # Replace my-gcp-project with your GCP project ID (same value as project_id above). image_registry = "us-central1-docker.pkg.dev/my-gcp-project/litellm/berriai" # Production: provide DNS already pointing at the LB IP for Google-managed certs. From 99e1a7612860fe469af44a5ef08434867359fd41 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 7 Jun 2026 02:10:22 +0000 Subject: [PATCH 11/12] docs(readme): equalize deploy button heights and fix Cloud Shell button font GitHub rewrites an image's height attribute to "height: auto; max-height: Npx", which only caps and never stretches, so each image renders at its intrinsic height. The AWS/GCP shields badges are intrinsically 28px while the Render/Railway buttons are 40px, leaving the row uneven regardless of the height="48" we set. Replace the two shields badges with committed 40px PNGs so all four header buttons render at the same 40px. Also swap the Cloud Shell button from open-btn.svg to open-btn.png. The SVG renders its label as live text with font-family "Roboto, Sans" and no generic fallback; since neither font exists in GitHub's render environment, the text fell back to a serif (Times New Roman). The PNG bakes in the correct typeface. --- .github/deploy-on-aws.png | Bin 0 -> 4113 bytes .github/deploy-on-gcp.png | Bin 0 -> 4922 bytes README.md | 10 +++++----- 3 files changed, 5 insertions(+), 5 deletions(-) create mode 100644 .github/deploy-on-aws.png create mode 100644 .github/deploy-on-gcp.png diff --git a/.github/deploy-on-aws.png b/.github/deploy-on-aws.png new file mode 100644 index 0000000000000000000000000000000000000000..06d41f2a5e06cd60aca118d53ea50f0beb234ce8 GIT binary patch literal 4113 zcmc)N)msyQy9aR4fiwf8hor>lPDgi1cS?-zZV)D&0wN;yqoh+A-5^Y20|!!~B9msL zk>h#JA8_u@xp|-O#rNX9df(@h`rJU1oP>!44-b!ATT9It4-Y^7Uw%&X;NN_|FByb~ zM>(ObrfeEsy8kY$fEL9vIS3bM#rAf(dhqKUl{=gEzT{xz0c&JDbyg^RA|A z>#0{c@dF@A;t4P<;5Jr^d|kPC(;i&d-MJopMA7#qYIAn2XMZj1J#2HgPtolV?pOTo z(^>yuD6HVNhm9!FG1-MEkyM2d7^DW&lgEk0P)T22RXMD6j5YVQ!!0YJT^zXB;tdTZ8* zD4<49vO2_+ndh7%j55ppVMME1G(lY;cVu6S^v@lvQ7K}$9#pFE$^_=`oE9UcrOv$O^Mlrb^^4B4RF*)+t?cj`jDyI!nJnnat_Z7h1kWFl&f}(kNUp~O8dRgU+ zO6w@AjOdeH3&L z5)BbCU{O2ad!451r>dcd#AKY}I$Hvz*rFg(*^`B-Ml20S?wFErtCL6GZyZB)k1`R$ zz3{#mabK0$y>p5NsU>RzmI?Y8k}ETK?L4#<4v!Hlgt(u3MDklhwhmA2qeGj}wR*JJ zw>4dQhp_w`Dz&=-=!Hx}kA`Q-GNLWm8^*K-GVuCa^Y^6;wF5rbjH0Vo4V7ekyk*H;9%y#v?mkN4Cx8rVD|c(VAcX&=xbF~r_}VEbT?{f{9-Z9SBJKb zv3?fslP_|o5ocv9O?b!6SzoXgc(7KCT&i`i&7HceUv9tL`&wss_h&waYzB3DU$_iZ z)Q5d|(S|EM!am+y#E ziJY)RwJy}c<;#*aCS>K>f0e?R6>RW6dtKCfsLixK_u2(J-Ywi3C;MHiav~lqt-P`_ ze-Tt5lu%&;nn7vQ?ut4R=IL0%(b2+4#NOp!2WwLO4!1!GI0~9A zl?@e)DssF!GKyxh&xX)EKuwbBrzUh5H9buuNsiZWzV!$#%D&F2Q560GlXr*0QP$9u zTGdN{x%BJE_nw z<*G&HB>dR*ZwSoPnxQp4Hr$Y)i3|a<@b%d6=y*5b zy{TOTqa|l4GZJzs{q%$G{+#fR^o`_R$dmQML++Dz76-TYJpxq(JzH&h1C=e?3g_G4pCIt8O}h2Ia#1?O>Q9RYVymRCHWOLHF+ z2!rt+HqniBVHp}Rc)CrF0FCa$7vdV^?P{0DP=1c9)?oW{>1 z=Jab1i1dm2^2(HA)h}8Dkz3u*DHh{8!o6Dn)5wJhvL* zXgCqA__uXA3#T2Ku%hPYGF>h6p&s!;bl7$;y6Elca5R>xP$6A)Goqv}K4w1r6-n&` z%i04q+=&LOY)EGxr4Osgr#xV=DARhp;7*l-OXv11blSvTRI#+D<+5XKc8Fwh4k1hx zbSc1iCaXEyo_i>?^PJyd;?PfIi(fy9f%25)VutFb=U0RzQzEyb$%|YeLpVZDdE6&Qq(~)dDT*TK$Ydjc2>eR=ioswRaII`1{XT__;Vfx22 z>1v967<=)3njZ-nG1J3U8FSarzWG&&Kv-(Lv@bP3CPdss%Ij}4 zh|Xc)F8u}|hol6uQ8cz3p{##Tl9&0Fobdl;sN&LUXT*Iw<)diA9l@Ohvb>kAM;fp>Jf-ET;a zb!Y_-ffGN9Sb%sc$|`8xqwIo{w<3lnm4p>NhNw^!3x?&uPxHr390O30qG;Wzd*3FZ zcr-kJ_T8OP$fERVjEhfSiqYAhpD^)RelKa50Ie&4E&L*Cu6Y)${HdSX$|OVQm@ zkVv1AwArVHGGOfq*;)o<=k*|KSyDP@Dp={!ptudlP{!9k%}e6rmN>xYiRvI9#Dgy@ zyLePDXZWO1>n?~A3eVM#zoOi^6uNII&QE)NrVg%tD`-#owA8jg{%Kbvl2hD}Mt|ZH zSW-FFxQ!hXnIT`3BlQ@$d{|tw&X-chSeF&=)bY}Jw(FT2g{rzl?1d1d$L3S2PEM}1 zMFV&YXv&+im}rS($~rpts1=@Qi}}gVP?r4ff~G2Q1`zNn5L$IJeJAEggV2ybb^^vf zLy|o1;4{ZrvKznQqtmLn*Ub3i$M2&m(26|eF5GSi3;c{JhSIE$KfYtBVifsAOmhod z&TXZwX_Mtegx2^pG;D!L89yhU4q(OU;9O+cn3f_VgA<#zgn_G-R_brtBVOL%a~(5{ zgjzR1{O|gok7^gqn6N@|y9R8+auKxV+h&YtYme$Cg~heQ$#(brP2* zXA5UCIdRerZU~U7OLvTKO7phMfV~nKaS^j$^RrU_(F0R8w4^1r06w?>*2oNKL$>L; zbpOrA56nVHtY!yF8W{uHBl%MQ2YbF;5;p3A>k8u?E!jPi z>)x07MgnDyh3i^IO_7olx;CRM&1txQ`+9o*UMW?G|6S3r_$WYSMZ!>SFdQM6COfNY zcK$KkJ@ZFP3T6sZp7Px6;`!i@vB(Yv8C@@JzLo>AR%$LKuBZWSB|Yy5-6$0! zPl~~q&j|H*vZ5<77(x+%Bb0jZoMj5B@tFHT!kWCC}*2>>Kr z2cZ$xyxk6-EUpY=Tm0ofo%79u>KEZ*NwV5ja8}7iKWPbXRcTbFH5Y7_d9Ii$sYU(i z4fE?m;@7*KkW^GpZ4!@}c48z2am)h=4RM zEU{HAKZY=*!h_$QX*P*6bSv9z+SIdu*ZMvlU$R+6$IsXp^N^kxq-TTeLUvyDyZW&+ z`1uA_EsqH3WqcS)AC8RiVR{cTd5l=v_l;596rg{VH~!_LnbFg>+h=Dr9IS;>j+cv@ zK)9QStF4}Q@~IaB+ysO_Gbj{gAn7vysyH&@QF5jk98=xKs!6RH4>fkfnxr=o&W)n{ zO6oxyWD{S6o)l(E#8j5;6bMCW|4C#|pPSv9DI%p*nIzYk$(T2LtBIV$HH%f;PXYY< zeXV=uXdbMXX{1O65+}!kNCXe~FfHL7=X-A^bCx{EUWXx+V$0Bz39+^=zg^b*C=T8) z&%9C*X>bi^zS+TeehnSI*5j>ySkT~XxrP5)nYo>+Iufh+q^FCH)d8F8MlB7n5dmWTv1S`V~^p^yps`DjgqhQJ2^8ej?xMn9{kjmn9UKJAcqM7? zTeOSSiiG4unxV!g7I%>!z2_age)xMl|1zjZ6D=B%pyBQ0U*Ky*AH&Yv&rqU}g4!8^ z{wakAG%`%jaK1ghTf;QriNdUMUA~M4f;Ib7MWOcIcRA(@h|(81cmb=&&(@AVirF>! z^~WYzrh94E_wf94 z7riyUDOdRF8LM)%HXeQ@PveH+fPu&D1SRr^q7TtTwbMBZCKF9BPWpaqlEpM|Qo3E$ zXB7@`*r>K|V~Y5cVkea~`lzU$*WOp@DLrd<^56tau(h_;UqhmF$~>LYnY4;$;scX( zMmjQXok8%#uu52bi(Iphdz)~erSY=Y8f|3T#@JK$CW9uz_xMkxEG%Y&c?rgG$n|rO zfPGuroOm5vR3~XZ?TU&NssXqE8au*Cw^ds6|9opY`+PeHH%RGi0nyTKgi=L2Rc0=X z>gXeNiWX^Kqo&pHs}NN7t_~yr1JtW*VuJrKmj3TL-nswh`TKh^iR+Sof$+aY{}2yI XH7&3dhx>acd4x^h`m+psv0$-R&i6K zu}5sGXdd^U@Vt1QH|ILnS=Twg>-XY&eG-fewCL{KyF)@kLZ_px4k95TZMgA`s3~u3 zX#4?ygoKeqM_t7fp0}HSJDp{^v!D1=VN&6Fq>M796DI}B3qHPk>_-0-GFiVZBG1H> zX`vtfv6W2Ncw~rA639~5M;hNnzEmH7NDv~Ae#-%oZomSpAXH9qb8VP*-|l;YCw@y6 zHH592zKigUsg*qY(#k(SsxPl+tRD3Wa%tZI#4!GP?!5FvF)|`rl^PK9o8u1WP0UUW zz<-nR5Q4-os)o|VA{kZVYd@lxRH+YNut?k#b|F+XsR3VSj0XQ}y7zzCRH_cdt;A8T z{w@G)&B+Q)Z*R`Y@@e_q)(z8x1nPqn=Rc$~NcTC7=V-pF7r)tE^6=MKHW_< zx>wFd^Uk={~e5?&eTCKVSyGe7%Ks0f${$W_-y6+C-U$!$NH7Z!lO_?dCmJ zX^<|!2I$GtG~VPyZGZ}v2{3fqbM7v7h? zr<%setjtq^vZlv2pq{|F3Z64jcYt-f0)TjBVeO+5n9^Mv|4n$GO^i2q+ZezZ;F)l; z!z9u1$1i|OduT`$%aAPLtAYO75E|Q~g;@Q!HuiaVm-hAbJKt*PYS5D>Fv91Nkh7t7 z@zc(vTr#l}nPT7k#PmFGb>}k3Vufaqb62>vTzPu$Sbuu+?^o9P!^CWBK?73b?t@|w zzKbUetWEzIdO^JHuh2d<>uGWv+K0e)G>VB0ORvhvd|)>1O&Od*bv>G}$ggzN7$y$L zk*HYJi;F=cxH6q~8b1tpE||C(nRv|eP&=tg5%em?yhL(j*zeB7?!M%{?Xj=okpa`-cpOD?=xedvYS!#OV_8@J_L71sJ5~K#vtAb(X}waa zjbYP)Tj}_~3%G}BSZY>>({aHjuHG-q{sF$u50Kv31-TBw+F{P=IH}gBp4z(Av8)Cv zB@9oyGi}bG*EY@+fSw{gyk0+Q8HHZoi*QzdG9l@GOeh}tTL|~Xg7GN&Cn*2pdJD5z zMonrbIi_Y%2~q!x3T8Do`Xyvx%YHbB{&1_P;04z++*4lAlgo6R?U|9j!PpK8ttRoe zD&DVFQi)x?*>I*1hBW`x22wDT{`<{xYP4S=KWdsPVx#feKFCc+&o)?4z?fm?dusuI z`{8c9pV08%kVYlX^f+hDwf1di8^wBav9LOGTz9ReS2jqn_TRdp?g+21aE=aDMKXp; z&jrqsqh+(T{vy0fiY!nl5Siktz$Vz^a44T)IQj{Wi8dOu8B&vJO>Lf7dR}eojuwqT zP%vodI0qh3Z8#@#uG#>1Qm}10#6KeT?)r{t?blx~=GvlCTgDzqa!u55dNosKc>XB)(5-sl7MdV&3e39Ai{z||i`Ra{ zWspV(0(|r6+7{*KXSJ|6tH)R}jJ$mTYdxk2G2gtG5qPmO>3{Tnnib)?h20dkw_41# z`n;2ePMI(=nGE#LjO3aTEk5l+=Jm7;t;wg(IMZ2e3MjB)^;J~U(b6kOulrjsqR6`Cn$4xS|$Si>53GsfhqfI zclnyjIM(-DUoJs4SLai6;`|HjxGsSkQ%6w1FKD%Ax}l=SJr!`bH0@Yk2N=tf&7I+} zG9@z$C{|>a9&#p+a z8mJ3_sF3G^>EbaRR>zI>c+!^*B{3BAy-Z>weWALT~`xn++pU-e(kQMg7on1@%y>@NS@?QhH+We=I6~1APR`moJfK4^Y+K$}TJR2AB_(;LjkNiX#mf zxpWE%U>&+mZa+;$61STvd%dD^tsGlvaxD3b9HrO2l_2+qgN{dH{pf0X9iPC6@0~H{ zpfP&R_}gkM^-8b>HW?q^WSH9~b6!jQfwD+T<<8@wl2<+nvFL;R#1PClmlct6#!p4M zv&Z=+Gq-2ZqV2}tS)>c9b)f1QGO=%h44pNvCyyNv@Zt#abux9=zWy?xf>H(Y#? ztMKVQuxItlc`H7s(?#x2aUGUBe%#=}yQB$Dron47c`gGvt=I$A92(;A+pq%_J5kCU1$nt{qdb*9u}g9#rd`3FCWq>BlbJ8rrmg&gxgv4TwQY9o;WnA? z<@IGc0*V|=GuuC0*rLnhOQK;pK;?3dpS9@M0KCAMF|m;= zTWtXb!rJJ!lHT2r%$PjZhN$Um$4`zC5*}`sf4FsCl^x2Yv>ZmFbxkbF;~9VGySNx= zr^ARabC{Z^7c!b)fWG(KY{bBLJZs9x^ZQDMZf1)HaiRFZLMkBa)KLmn8nR;6xFNYZ z{$`!KyUJ=T38G(MNxDzl$z@&zs;M-Dj{#Fv#R?AB$Xbys8gj;{VW#Sra;iT9N?!IHQfIf~H^*WBnpVIeM zR4?ic%jEFOThZYT!mE{>TIpQqe|Wki`ONW~sj`_LC`>NY2jBV>0Y^0pB!z99;ewwB zY*w%}e*9pSbMOhJq~aJLt^eu}k&jE{s9F9VkUbM6;3vOZIZ5O>(w_^k26v;&wfU?l zW|0r#SfXQC$bg69w z+OMdP>~`qxY^1Fm_n^Lhj#((#a5L=L1vLFVx=i>qnSC%r)Dpiy;3k==rTI9WMSZ37 zgjUw4@WNWltt$iNFEqx5a)AG87dEFsAVOQ5iubW@_@1~X+VTEP;0A!)zQ!`tl3a;6`$6_u?hZ|dqOXr!NLe!CW5!B z0QSzSrvI`O=`u55`%BN3ce{g|RptYyxIk4fKBMI`nN4`8!PKlkIj2YyKNo@c6}zcf zI^_H~Om6P4?)g=~7HX+Ak>X!E0o;l6mV&n@0wO%2yrjL&270$4wfEPXn6aj*M5eym zs&O_-`Y;vt`j9v-bX=;5{VtR>yHPcs$sRVPRxMm2O>-#u4$AvVTv|Yb=UDcU!)Ctk zJtS4m*6`#l@&L6aGwzeKr2?)mD%9ddhIgkE%lZ89rZJysZiGn3xZ8fe=@}_)O_f=D?j*jXKWQ%u%O-m=X7#`!O{BmwZJ`2j{iY z+QpK5PepKJm{{Sfr1H}VV*eeJm&{>$Li6Wx{QA1N^|xzT`w0A#`P@*#Vk>cZ3c6R? z6YD%#6En{IPIswStsm?p?6+C_I774i6v5rP#&9=>S=Ppx5DU}ZC~racTSM9IvZNxp zHa%C#gXT{<*!yqO;owowC+7yxXY`Q>;s+$ITAM<1)FQY=()HQj&HY@n&z&#)`*;ot zq)S)JTHU-03X7+oSj)U|+krmxqW=S{;i!nf3$L$HRh@6v#{dy-$ZUDydXt*xF_La z1?7Yx!|C?9Vh~sEFO=jAY%*UzL2Ru1T^9{VYnxE2rNe0-Eq||LLK#zr-tO3t8f5s^ zEINl5q_8d~YGI-a-hV)fO$M7o%lZ)eaYNiEI$OXLU&>ccvj$JM3B5b}A(Au&6w>`Y zxB%Sv`4=vV+GbaCV+{6n}1QUMgx87pb4UVN*LW{CbAmM&`b}$U=lh z5F#^$2Wy2(80^cbj{MGb)#AhxJ`hV`%S{y@Z&qSguM+-yx+HsJpz&hRpS9mhB-Xlh z|Ru&Acapd_PSpRn=xr`RHQh(AHtn6=8 zU@U8PvRD1{J&KTjt3QdwzrAN!aY3nfNvz&tV5RSoc<=Errl*aOPcj9zg`VRJy2`e9 z)Ov4d8t#!VLAhO|^O%7OIT~@i@+PAe!4%oTTMVjDB!uwu_y4eyH}&$$%o&@NtF&_Z z4(|pZm(tIl{b})OWt3S@+iRG0!&lh%?JdX6D>T(!kk2;^P|c07pbGmIlU@SbF6!+* z=c%i8)JRj(P@k!M&COZi-eH4|xUxLyY#FyZ7UZr?AyX~z(&SoZHOR7rni94m0})_y z^Gyj0{BKEsYuVDFKeC|s6Xq>ZZoRd-bIwD|^mQRuE95L~zLH^e7j#9EC>nEV-K*EuA= zUIQdnXoIg|-|=<7eJEke9XIn{x~+{y@qxySlDS8d?kX8F+G%I+S~L;TrBs58k6g_m zl)Z7C=gD^bsNolD_u4p!JPDsVV@YEm{yF-WW^?vjQ#9C~iuB}s*8*S&juvvU#r)Gl zVk@O3YAy9^3_{zA_MXA+~u-6(lve$4kwGi^MFX1IT5p`ZDNJLbrH{#*dkc$BBK zcrOq_#qsp{<)_lGHE2~te!XBS6caXa$(*aUO0-SX{E)eq{$MVz

Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.

- Deploy to Render + Deploy to Render - Deploy on Railway + Deploy on Railway - Deploy on AWS - Deploy on GCP + Deploy on AWS + Deploy on GCP

LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website

@@ -467,7 +467,7 @@ Provider API keys live in AWS Secrets Manager; reference ARNs via `gateway_extra #### GCP — Cloud Run + Cloud SQL + Memorystore + HTTPS LB -[![Open in Cloud Shell](https://gstatic.com/cloudssh/images/open-btn.svg)](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FBerriAI%2Flitellm&cloudshell_workspace=terraform%2Flitellm%2Fgcp%2Fexamples%2Fdefault&cloudshell_tutorial=TUTORIAL.md&cloudshell_image=gcr.io/ds-artifacts-cloudshell/deploystack_custom_image&shellonly=true) +[![Open in Cloud Shell](https://gstatic.com/cloudssh/images/open-btn.png)](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FBerriAI%2Flitellm&cloudshell_workspace=terraform%2Flitellm%2Fgcp%2Fexamples%2Fdefault&cloudshell_tutorial=TUTORIAL.md&cloudshell_image=gcr.io/ds-artifacts-cloudshell/deploystack_custom_image&shellonly=true) Real 1-click. Opens Cloud Shell, clones this repo, and walks you through `terraform apply` via a built-in [DeployStack tutorial](./terraform/litellm/gcp/examples/default/TUTORIAL.md) — pick the project, the tutorial sets up the Artifact Registry remote repo, writes `terraform.tfvars` from your answers, and runs apply. From 7cf1b263356fa6960da275f360db0031182fb1e2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 7 Jun 2026 02:10:22 +0000 Subject: [PATCH 12/12] docs(readme): collapse Railway deploy anchor to a single line The Railway button wrapped its img across indented lines, so the anchor contained leading and trailing whitespace. GitHub underlines link content, rendering that whitespace as a small blue underline beside the button. Put the anchor on one line like the other three buttons so there is no inner whitespace to underline. --- README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/README.md b/README.md index 9e6b4030966..719f74e5924 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,7 @@

Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.

Deploy to Render - - Deploy on Railway - + Deploy on Railway Deploy on AWS Deploy on GCP