← All case studies Terraform → Pulumi

Starburst's Pulumi page says Galaxy deploys fell from two weeks to three hours

Pulumi's Starburst case study quotes a two-week Terraform deploy and "about three hours a day" after. The same page also says three hours flat, 112x, and a 99% cut. A 2023 Pulumi blog about the same engineer says 3x and a few minutes. Those are different sentences.

This is Pulumi’s page, Starburst. The canonical URL is /customers/starburst/. Read it as a vendor case study. Matt Stephenson, Senior Principal Software Engineer at Starburst, is the person they quote. The page does not give a publish date. A second Pulumi page, from August 2023, summarizes a talk he gave, and its speed claims do not match this one.

Galaxy is their managed Trino service. It runs in 22 regions on AWS, Azure, and GCP, across 50 to 60 Kubernetes clusters. The control plane is mostly EKS. Data plane services run on the managed Kubernetes of whichever cloud and region that cell uses. Each service is a Java Airlift app on a cluster. A platform team owns the infrastructure. A service team ships Galaxy on top of it.

The Terraform → Pulumi playbook is state, providers, and secrets. These pages are a vendor’s speed claim, and the claim is worded more than one way.

Read the quote, then the headline

Stephenson says:

When we did it with Terraform, it took two weeks to do [infrastructure deployments]. Now we do it in about three hours a day.

The headline and the key stats say something tighter. Two weeks became 3 hours, and they call that 112 times faster. The body uses the same three hours for Galaxy’s multi-region blue-green deploys, and it credits full automation of the pipeline, so nobody runs glue scripts by hand. The closing section also says a 99% reduction in deployment time.

If you treat two weeks as 336 hours and divide by 3, you get 112. One minus 3/336 is about 99%. The page never shows that division, and it never says the quote’s “about three hours a day” is the same event as the headline’s three hours. Only the flat three hours produces 112 and 99%. There is no dollar figure next to any of them.

What Terraform was doing

A Galaxy deploy was blue-green. New Kubernetes clusters in every region, new services, then delete the old clusters. With Terraform that took two weeks, because of glue scripts and manual runs. GitHub Actions was hard to hang off Terraform, so scaling to another region stayed manual. HCL made it hard to hide the differences between EKS, AKS, and GKE behind one abstraction. Autoscaling, upgrades, and node groups were a different implementation on each cloud, and a domain language does not give you an object to hide that behind.

Service developers filed tickets. A review environment could take an hour. A Terraform template could build the base, and each environment still needed its own config. There was no path through GitHub Actions for that leftover.

What they run now

Pulumi programs are in a monorepo. Stephenson’s team used Java, so clusters and managed services became ordinary objects, with shared functions for autoscaling, upgrades, and node groups. A base cluster could be configured for the shape a cell needed. Deploys go through the Automation API, triggered by GitHub Actions. A service developer changes a list and opens a pull request. The workflow creates the infrastructure.

In 2023 management wanted to cut cost, and they left Pulumi Cloud for a state bucket they ran themselves on S3. Replacing the cloud product meant building state, a dashboard, insights, and deployments themselves. That took, on average, two engineers for one or two days a week. They came back. The page says the cloud product pays for itself because the dashboard shows unused resources they can turn off. Still no dollar amount. The tradeoff they describe is time on a homemade control plane versus time on Galaxy.

Several later gains are plans. They intend to run Pulumi Deployments so a lost GitHub Actions Spot instance stops interrupting an update. They had started using Spot for those runners. They say a test stack that took an hour will take half that with review stacks and a TTL. Developers had been running Galaxy locally in Docker, and an end-to-end test there was not keeping up, so the cloud test stack was the thing they wanted torn down after the review or after a timer. They plan to use ESC so developers stop copying keys out of KMS. Drift detection is described with an on-call story about a manual change that would break the next release, and an alert that lets them fix it first. There is no incident count. Pulumi AI, they say, turned an AWS VPC and EKS rewrite that would have taken a couple of months into a week or two. A stats line calls that a factor of ten. Same rewrite, two phrasings.

The other Pulumi page

On 23 August 2023 George Huang summarized Stephenson’s PulumiUP 2023 talk. This is also Pulumi’s writing, a summary rather than a transcript.

The prior stack in that summary is wider than “Terraform was slow.” Terraform for infrastructure, Argo CD for Helm, Maven, Docker, and a Slack bot, tied together with GitHub Actions on specialized runners. Wrappers around the old tooling made the pipeline hard to keep. Clusters were scaled by hand as load moved. A Trino upgrade could interrupt customers.

Huang gives two reasons Stephenson brought Pulumi in. The languages were Java and TypeScript, which the engineers already wrote, so infrastructure and application code could sit in one place. The Automation API let them drive the engine from their own programs and from CI, without another wrapper layer.

The speed line on this page is different again. Deployment of new customer releases got 3 times faster. Blue-green updates of the data plane finish within minutes, with zero customer downtime, and customer runtimes, including VPCs and Kubernetes clusters, update twice a week. The integrations that move a customer onto the new cluster are internal APIs, not a button in the case study. The same summary says one tool manages over 50 clusters and the Helm releases on them, plus Cloudflare, Confluent, and Datadog.

Three times, a few minutes, and twice a week do not convert into 112 times or into three hours. The case study and the talk summary can be about different slices of Galaxy. Neither page says which slice. I would paste Stephenson’s sentence, the 112x line, and the 2023 “within minutes” line as three quotes.

The playbook will not tell you which one your pipeline will hit. A rehearsal on one region will, and it should time the same kind of deploy you are quoting. A blue-green of every region is not a review stack, and a talk summary is not the case-study headline.