Static Website Playground: Terraform on AWS

Tony Morris — Tue, 31 May 2022 17:32:17 +0000

I need a side project, so here I am.

I make a lot of strong statements about my opinions of public cloud providers and infrastructure-as-code tooling. I figure I may as well back them up with real actual projects.

This is the first post in a series that I'm calling Static Website Playground. Each post will have the same premise: build a static website and deploy it to a specific public cloud, utilizing some type of infrastructure-as-code tooling.

This post is about using Terraform to deploy to AWS.

The important piece first:

URL: https://aws-terraform-static.morriscloud.com/

I was going to draw a diagram to represent the architectural layout here, but... it's super simple. So I'll just discuss the interest bits below.

GitHub

URL: https://github.com/morriscloud/static-website-playground/tree/main/terraform/aws

Each of these static websites is stored in the same repository and organized by IaC provider and then by cloud provider.

Cloudflare

Every post in this series will be using Cloudflare as the public DNS provider. I've got a Zone that I created previously (morriscloud.com), so this part is super simple. I look up the Zone using a data resource, then I create a new CNAME Record in the Zone that points to the S3 bucket's website endpoint. More on that part later.

Note that I recently had to update this to use the new S3 resources in the Terraform AWS provider, so I'm now using aws_s3_bucket_website_configuration.this.website_endpoint as the CNAME target, rather than the previous aws_s3_bucket.this.website_endpoint.

AWS

This deployment is as bare-bones as possible in AWS. It creates an S3 bucket, uploads an HTML file index.html, and sets some properties on the bucket to enable it to be viewed as a public website. There's a small bucket policy that's also attached to the bucket, allowing GetObject access to the objects in the specified bucket from anywhere in the world.

Terraform Cloud

To deploy the resources, I use the free version of Terraform Cloud. I connect a workspace to my GitHub repository, then I use the working directory to point to the correct subdirectory in the repository.

Using this project as an example, I have a workspace named static-website-playground-aws-terraform, pointed to the GitHub repository linked above, and then set to terraform/aws as its working directory.

Using Terraform Cloud gives me a nice hook directly into Pull Requests (if you enable it), plus I can see the changes that I'm making in a feature branch easily.

It's important to note that my favorite feature of using Terraform Cloud is the ability to run terraform plan from my local console pointed to the remote workspace. This runs the plan in a Terraform Cloud runner with all the variables defined there, rather than somewhere on my laptop that I could accidentally commit to source.

What's Next?

Next up will be using Pulumi to deploy to AWS. Obviously it won't have Terraform Cloud, but I imagine most of the rest of it will be exactly the same. Stay tuned!

Don't Manage Terraform Enterprise With Terraform

Tony Morris — Thu, 26 May 2022 15:13:07 +0000

Don't do it. I know you want to, and you should not.

First, some facts.

Terraform Enterprise is a wonderful product. It's the self-hosted distribution of Terraform Cloud for organizations that want the privacy and scale of an enterprise-grade installation.
There exists a Terraform Cloud/Enterprise Provider that can easily template and manage how an organization creates Workspaces (and other TFC/TFE resources).

Given the generally painful user experience of entering dozens of Workspace Variables into the TFE, it makes sense that nearly everyone I've worked with has stated a desire to use the tfe provider to manage workspaces. I'm here to tell you why this ends up being a rougher idea than you hoped for.

Pricing

Terraform Enterprise is priced by the number of Workspaces you have.

They're not cheap.

If you start dedicated Workspaces to creating and managing other Workspaces, you're effectively shorting yourself out of your own licenses.

On the other hand, if you're in Terraform Cloud, you're not paying per-Workspace, so feel free to use this method if the following hiccups don't pertain to you.

For what it's worth, if HashiCorp had a concept of "Configuration Workspaces" that didn't hit against your Workspace count, then this would obviously not be an issue.

Multi-Step Updates

Let's say you can get over the pricing issue. Talking through how the Configuration Workspaces would be architected and deployed brings up some other potential issues.

First, a quick overview of how we had tried this out.

The Setup

Workspace Repository

The most important piece of functionality here is the Terraform module that uses the tfe provider to create the Workload Workspaces.

Everything about the Workload Workspace is contained within this Workspace Repository, including the Workspace Variables.

Another name for this repository could be a "Configuration Repository," as it configures the implementation of the Workload Repository.

Workload Repository

The Workload Repository defines the Terraform resources to create the workloads that you are configuring. For us, this is a bunch of resources from the aws provider, but it could be anything.

Configuration Workspace

The Configuration Workspace in Terraform Enterprise is pointed to the Workspace Repository. When it executes a Run, it generates Terraform Enterprise resources, such as the Workload Workspaces and the requisite Workspace Variables in each one.

Workload Workspace

The Workload Workspace is created by the Configuration Workspace and pointed to the Workload Repository. When it executes a Run, it generates Workload-specific resources. In our cases, this is generally AWS resources such as EC2 instances, EBS volumes, etc.

The Problem

The biggest problem with this setup is the back-and-forth you have to do in order to make changes.

Let's say you want to add a resource to the Workload Repository. This is straightforward, and it doesn't cause many issues. You would just commit your changes to the Workload Repository, and the Workload Workspace would pick those up.

What if, however, that introduces a new variable to the repository? The changes you would have to make in order to get it through the system look something like this:

Add the Workspace Variable resource to the Workspace Repository.
Push the Run through the Configuration Workspace to add the TFE Workspace Variable to the Workload Workspace.
Add the variable to the Workload Repository.
You can finally push the Run through the Workload Workspace.

Four steps for a simple variable addition seems like a tough solution to roll out to any team.

What Should You Do Instead?

Let's be honest. I'm certain there are some really creative ways to work around these limitations managing Terraform at-scale. Feel free to use them and tell me what they are! Hit me up on Twitter if you find a really neat solution!

For our use cases, we are building out a Control Plane that abstracts that business-level functions from TFE itself. So, instead of thinking about "I need to create this TFE Workspaces with these Workspace Variables," we are now thinking "This team needs to create this application cluster."

This abstraction is not unique. I've talked to many people in the community that do a similar thing.

This solution works really well for us because we have a number of backend systems that we need to integrate together during an "application cluster spin-up." These include SaaS tools, such as PagerDuty, Splunk, New Relic, and many others. Given that Terraform Enterprise has a robust API, it makes our Control Plane much more straightforward to implement.

The Ops Community ⚙️: Tony Morris