The Ops Community ⚙️: Xavier Mignot

Azure App Service, deployment slots (with configuration !) and Terraform

Xavier Mignot — Mon, 26 Dec 2022 22:35:22 +0000

When you want to implement blue-green deployment using Azure App Services, deployment slots is the way to go. You can create a staging slot to deploy the new version of your code, test it and swap with the production slot once ready to make the new version go live. All of this without causing downtime.

Another main feature of App Service is configuration with app settings, who are environment variables set at the service level and injected to the application code.

I have been using all of these for several projects over the years, but once we have started using IaC (especially Terraform), things became complicated as swap operations were messing up with the Terraform state.

After almost stopping using slots over the last few years, I have finally found an approach to make them work using Terraform and I'm happy to share it in this post.

By writing this post I assume you already have a good understanding of Terraform and App Service, at least you have tried to use them together 😉

If this sounds new to you I recommend reading my first post about Terraform:

dev.to

as well as some content on App Service (see the overview, app settings and deployment slots pages from the official documentation).

The problem with deployment slots and Terraform state

The main issue with deployment slots and Terraform is that the swap operations are usually performed outside of Terraform. After a swap not only the deployed package changes from one slot to another, the swap impacts also all the configuration (including app settings), application stack, Docker image if you are using containers, etc.

All of these properties are included in the Terraform state, so the the next apply after a swap will try to revert the changes, probably causing a mess.

You might disagree on the "usually performed outside of Terraform" part of this paragraph. I know there is a resource in the AzureRm provider to perform swaps, I have tried and abandoned it and explain why later in this post.

TL;DR: The solution, "briefly" explained

If you are here only for the solution and don't want to check the demo and read the whole thing (which is fine), I'll try to explain my approach as briefly as I can in this section.

The solution relies on the following principles:

An active_app input variable to the main module whose value can be either blue or green. The main module also receives two app settings variables: one for the blue version of the app, one for the green one. These 3 variables are used together by the module creating the App Services resources in the following way: the settings of the active app go to the production slot, the other to the staging slot.
The active_app variable is also set as an output of the main module. This might sound silly but it's the way I've found to store the following information in the state: What was the active app during the last terraform apply, the blue one or the green one ? And this is a key part of the solution.
The swap operations are made using the Azure CLI with the az webapp deployment slot swap command. Right after a swap, the previous active_app value is retrieved from the state using the terraform output active_app command. Then a terraform apply is made using the new value of the active_app variable (which is the opposite of the previous one). This apply does not changes anything to the infrastructure, it changes the value of the active_app output, thus saving in the state which version of the app is live.
Before applying any change to the infrastructure that is not related to a swap, the latest value of the active_app is retrieved from the state (still using the output command). The same value is passed as the active_app variable so that the properties of the slots are not swapped by the apply command.

This is it, it was not really brief but those 4 points need a minimum amount of details. Still confused ? I got you covered, the rest of this post shows a demo with more in-depth details.

GitHub repository

As I almost always do I have prepared a GitHub repository with a demo consisting of a simple ASP.NET web app, Terraform code and GitHub Actions for deployment:

xaviermignot / terraform-app-service-slots

A demo for using Azure App Service deployment slots using Terraform

Azure App Service with Terraform, deployment slots and app settings

This repository contains a demo on how to use Azure Services deployment slots with Terraform (yes, with app settings too !). It illustrates a blog post published here.
The demo consists in :

a few Terraform files to provision an App Service and a deployment slot
a simple ASP.NET web app
GitHub Action workflows to
- provision the Azure resources
- deploy the web app: the blue version on the production slot, and the green version on the staging slot
- swap the staging slot with the production one (as many time of you want)
- destroy the Azure resources once you have finished to save costs

Getting started

To run this demo by yourself there is nothing to install on you machine as everything is running in the cloud. You need to configure a few things in GitHub and in Terraform Cloud.

…

View on GitHub

You can fork this repo and run the demo by yourself by following the instructions in the README file. Or you can stay here, I will explain how things work in the next section.

The demo, in action

The demo consists in a simple ASP.NET application (running in an App Service of course) presenting a page that can be either blue or green.

The structure of the repository is very classic:

The src folder contains the application code
The infrafolder contains the Terraform files
The .github/workflows folder contains the GitHub Actions workflows

I took this demo as an opportunity to level-up my GitHub Actions skills but I won't focus on it, as everything can be done locally or from any CI/CD solution.

Provision resources, and deploy code in both slots

Let's start when the blue version of the application is already running in production, and the green one being tested in the staging slot.
To fast-forward to this situation, I run the initial-deployment workflow of the demo to perform the following actions:

Using Terraform, create a resource group, an App Service Plan, an App Service and a staging slot.
Checkout the blue tag of the repo, build the code and deploy the package to the production slot of the App Service
Checkout the green tag of the repo, and also build the code but deploy to the staging slot

So far this is what we have in production:

And in staging:
You see that red panel over here ? Its presence is managed using deployment slots (or sticky) settings

A little glimpse inside the IaC

As you can see in the pics both versions of the app shows a value in italic that comes from configuration, and a red panel only on the staging slot. This done using the active_app variable declared like this in the main module:

variable "active_app" {
  type    = string
  default = "blue"
  validation {
    condition     = var.active_app == "blue" || var.active_app == "green" || var.active_app == ""
    error_message = "The active_app value must be either 'blue' or 'green' (defaults to 'blue')."
  }
}

In the call to the app_service module, this variable is passed alongside the settings for the blue and the green app:

module "app_service" {
  # ...
  active_app = var.active_app

  blue_app_settings = {
    "EnvironmentLabel" = "This is the blue version"
  }

  green_app_settings = {
    EnvironmentLabel = "This is the green version"
  }
}

In the app_service module, the active_app variable is used which settings should be used for the production slot (which is the App Service resource itself):

resource "azurerm_linux_web_app" "app" {
  name                = "app-${var.project}-${var.app_name}"
  # ...
  app_settings = var.active_app == "blue" ? var.blue_app_settings : var.green_app_settings

  sticky_settings {
    app_setting_names = ["IsStaging"]
  }
  # ...
}

Notice the sticky_settings block ? It is used to display the red panel only on the staging slot, whose creation also relies on the active_app variable but in a slightly different way:

locals {
  # Store the "non-active" app settings in a local
  slot_app_settings = var.active_app == "blue" ? var.green_app_settings : var.blue_app_settings
}

resource "azurerm_linux_web_app_slot" "staging" {
  name           = "staging"
  app_service_id = azurerm_linux_web_app.app.id

  # Combine the "non-active" app settings with the sticky one
  app_settings = merge(local.slot_app_settings, { IsStaging = true })
  # ...
}

Let's swap !

Swapping is done using the azcli-swap workflow who first runs the following command:

az webapp deployment slot swap -g $RESOURCE_GROUP_NAME -n $APP_SERVICE_NAME -s staging

And just after that it saves the newly active app in the Terraform state using this script:

terraform init
# Get the previous active app from the state...
currentActiveApp=$(terraform output -raw active_app)
# ...and "reverse" it: green turns blue, blue turns green...
[[ "$currentActiveApp" == 'green' ]] && newActiveApp="blue" || newActiveApp="green"
# ...finally save the new active app in the state
terraform apply -auto-approve -var active_app=$newActiveApp

Which produces the following output:

Changes to Outputs:
  ~ active_app = "blue" -> "green"

You can apply this plan to save these new output values to the Terraform
state, without changing any real infrastructure.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Back to the browser, the green version is now in production:

And the blue one has moved to staging:

In case of a rollback...

To perform a rollback, just run the azcli-swap workflow again ! Executing the same steps will put the blue version back in production, and green one in staging, just like before the first swap.

In the real world, this is why it's handy to keep the previous version in the staging slot: even if the new bits have been tested in staging, problems can still occur once in production so better be ready to put things back in place !

What about other changes to the infrastructure ?

To apply changes not related to a swap, it's almost as simple as your average terraform apply. You just need to retrieved the current value of the active_app output and pass it as a variable:

activeApp=$(terraform output -raw active_app)
terraform apply -auto-approve -var active_app=$activeApp

In the workflow code, the output is done like this: activeApp=$(terraform show -json | jq -r '.values.outputs.active_app.value // "blue"')

This is to handle the first apply, as the output -raw command returns an error if the state is empty. I have filled an issue for this, any 👍 is appreciated 🤗

A few words about the `azurerm_web_app_active_slot` resource

Before closing this post, I want to talk a little about another solution I had tried initially but later abandoned.

The AzureRm Terraform provider provides the azurerm_web_app_active_slot resource to perform a swap using Terraform.

There is even a page on Microsoft Learn that explains how to use it and has inspired me to write this post.

Initially I had built my demo using this resource like this:

resource "azurerm_web_app_active_slot" "active_slot" {
  count = var.active_app == "green" ? 1 : 0

  slot_id = azurerm_linux_web_app_slot.staging.id
}

And it worked: applying the configuration with the active_app set as green did perform a swap.

Things got weird when I tried to rollback: applying again with active_app as blue removed the active_slot resource from the state, but it did not make a swap in the other way. Actually I had to apply the configuration again with active_app as green to make the blue app active again 😖

And it went crazier when I put app settings in the mix: I always ended up with the blue app settings attached to the green app, and vice-versa...

Finally I think that this resource performs an API call to make the swap, aka a one-shot operation that will make changes on the infrastructure described else-where in the code-base.

The result is a mix (a mess ?) of:

an imperative approach, aka this active_slot resource who tells "do this swap operation !"
a declarative approach , aka the rest of my Terraform code-base, who tells "hey I need this stuff but I let you do your thing to make it happen"

And I prefer not to mix both approaches, that's why I ended up separating them and notifying the declarative stuff (the Terraform state) of the changes made by the imperative stuff (the swap with az cli). Damned I really hope this last sentence makes sense 😅

Wrapping up

Getting to the end of this post has been quite a ride, the writing went pretty smooth but once again I have probably spent way too much time on this demo. But I'm happy with the result, I hope this provide one approach that everyone can use as a starting point to combine the benefits of deployment slots with Terraform.

I have learned a ton doing this, as I had barely touched GitHub Actions before. There was also a few gotchas in Bash scripting, and I can't remember the last time I built even the simplest webpage without a frontend framework...

Thanks for reading, feel free to reach out to me if you need, and happy coding 🤓

Deploy Azure Logic Apps as code

Xavier Mignot — Fri, 26 Aug 2022 13:57:15 +0000

I have been playing with Azure Logic Apps on my own lately, and was wondering how these can be managed by teams in an enterprise environment, especially how can we automate the provisioning and deployment of Logic Apps.

In this post I will show how we can do this using infrastructure as code using Bicep.

Presentation of the context

As an example we are going to build a very simple project consisting in the following elements:

A Storage Account
A table in the Storage Account
A Logic App triggered by HTTP: each request will add a line in the storage table with the IP of the caller

The following diagram shows what we are going to create here

GitHub repository

I have prepared a GitHub repository to let you see the whole code of the demo, and eventually run it by yourself.

The repo contains the full IaC code in Bicep and Terraform, in the post I will show the most relevant parts in Bicep only for clarity reasons (and also because it's all little bit more complicated in Terraform, more on this later).

Creating the base resources

To get started we are going to create all the resources but the Logic App itself. If you're not familiar with Bicep this will help to understand the rest of the code, otherwise you can jump to the next section.

Note that my naming convention is composed of the same suffix in all resources names (with a random part to ensure the unicity of the name), and a prefix depending on the resource type.

You can use this page from the Cloud Adoption Framework as a reference for the abbreviations for Azure resource types.

This periodic table of Azure resource types is also pretty neat.

I'm new to Bicep but already used to create a main.bicep module containing a subscription deployment to create the resource group, and a resources.bicep module containing a resource group deployment.

I will focus on the resources.bicep module here, here is the beginning of the module with the creation of the Storage Account, and the storage table:

@description('The suffix to use in resource naming.')
param suffix string
param location string

resource storageAccount 'Microsoft.Storage/storageAccounts@2021-09-01' = {
  name: substring('stor${replace(suffix, '-', '')}', 0, 24)
  location: location

  sku: {
    name: 'Standard_LRS'
  }
  kind: 'StorageV2'
}

resource tableStorageService 'Microsoft.Storage/storageAccounts/tableServices@2021-09-01' = {
  name: 'default'
  parent: storageAccount
}

resource storageTable 'Microsoft.Storage/storageAccounts/tableServices/tables@2021-09-01' = {
  name: 'logicAppCalls'
  parent: tableStorageService
}

Creating the API Connection

To interact with a resource or a service, a Logic App requires an API connection, which is basically a wrapper around an API, the Azure table storage API in our case.

This is one of the key steps of this post, as the service connection contains the way the Logic App authenticates to the Storage Account. I could have used an account key but decided to use a managed identity to follow best practices (this post helped me a lot to make this part work).

Long story short, the trick with the Managed Identity is to use the 2018-07-01-preview API version of the Microsoft.Web/connections resource type. Bicep will show a warning because of this but it's currently the only way to make it work:

resource tableStorageConnection 'Microsoft.Web/connections@2018-07-01-preview' = {
  name: 'tableStorage'
  location: location

  properties: {
    parameterValueSet: {
      name: 'managedIdentityAuth'
      values: {}
    }
    api: {
      id: '${subscription().id}/providers/Microsoft.Web/locations/${location}/managedApis/azuretables'
    }
  }
}

Creating the Logic App workflow

Why are Logic Apps different ?

Before jumping into some more IaC code, let's talk about how teams could manage (develop, version and deploy) Logic Apps and why they are different from other Azure resources like Web Apps or Azure Functions.

Functions and Web Apps are code-first services whose deployment require two steps:

A provisioning step to create the resource in Azure (using IaC, the CLI, the portal, ...)
A deployment step to push business code to the resource (using a CD pipeline, VS Code, a git push, ...)

Basically an empty shell is created, and then content is sent into the shell. Both steps can be achieved by different teams using different technologies.

Also deploying a new version of the business code does not reflect any change on the infrastructure.

Logic apps is a design-first service, typically you design your workflow in a GUI (like the Azure portal), and it is saved in a JSON "code" tied to the Azure resource.

So there is not the same separation between provisioning and deployment as above: any change to the business logic will be reflected on the infrastructure.

In other words, the "code" or the "logic" of the Logic Apps cannot be deployed separately from the creation of the resource itself.

Let's develop/deploy this Logic App for good now !

So how can we develop Logic Apps from the GUI and automate their deployment using IaC ?

Here is what I've got in the Azure portal once I've finished "developing" my Logic App:
A similar experience is available using the VS Code extension

Switching to the Code view allows me to see my workflow in JSON. From there I can copy the definition element and save its content a file in my repo.

Why not taking all the content of the code view ? Because the parameters element contains the resource id of the API connection (with the subscription id and resource group name) and the Storage Account name, and I don't want these kind of environment-related information to end up in my git repository.

Now that my logic is saved (and versioned) alongside my IaC code, I can use it in the definition property of my Logic App workflow (using the loadJsonContent builtin function):

resource logicApp 'Microsoft.Logic/workflows@2019-05-01' = {
  name: 'ala-${suffix}'
  location: location

  identity: {
    type: 'SystemAssigned'
  }

  properties: {
    definition: loadJsonContent('../logic_apps/insertIntoTableStorage.json')
    parameters: {
      '$connections': {
        value: {
          '${connectionApiName}': {
            connectionId: tableStorageConnection.id
            connectionName: connectionApiName
            id: '${subscription().id}/providers/Microsoft.Web/locations/${location}/managedApis/${connectionApiName}'
            connectionProperties: {
              authentication: {
                type: 'ManagedServiceIdentity'
              }
            }
          }
        }
      }
      storageAccountName: {
        value: storageAccount.name
      }
    }
  }
}

The only thing left to do is to grant the Logic App access to the Storage Account using an assignment to the Storage Table Data Contributor builtin role. This can be seen here in the repository.

Finally everything can be deployed using a simple az deployment sub create command.

What about Terraform ?

As mentioned earlier, this little demo project has been done in Bicep and Terraform, but I ended up keeping only the Bicep version in the blog post. This is for clarity reasons but also because doing this with Terraform has the following drawbacks:

There is no support for API connections in the AzureRM Terraform provider, so I had to create an ARM template and invoke it in my Terraform code to create the API connection
A Logic App workflow can be created using the logic_app_workflow resource of the AzureRM provider, but it doesn't provide a way to set the definition of the workflow. The solution was to use this resource to create the workflow (with the Managed Identity for later role assignment), and then another ARM template file to deploy the definition of the Logic App from the Terraform code.

Wrapping up

This post shows a way to achieve a first step in managing Logic Apps at scale.

There are still many things to cover such as how to run the deployments in a pipeline, how to make a change and bring it from development to production, how to allow changes using the GUI only in development and force the use of pipelines in other environments, etc.

I hope this post will help you to bring your Logic Apps to the next level, do not hesitate to reach out if you need, and thanks for reading 🤓

TLS with Terraform and Azure: use managed certificates

Xavier Mignot — Wed, 25 May 2022 19:52:41 +0000

This post is the last one of my series on the generation of TLS certificates with Terraform for Azure, after the post about self signed certificates and the one about Let's Encrypt.

For this one we are going to let Azure manage everything by using managed certificates, a feature available on several services that let Azure handle the generation and the renewal of certificates.

Previously in the "TLS with Terraform and Azure" series...

If you haven't read the previous posts of the series you can check out this section of the first one to get the common context for the whole series.

There is also a GitHub repository that contains all the code from this series of posts:

xaviermignot / terraform-certificates

A repository showing how to generate certificates using Terraform

Certificate generation with Terraform for Azure App Service

This repository contains sample code to generate TLS certificates using Terraform.
It uses an Azure App Service as an example of a website to secure.

The certificates are generated in 3 ways:

By creating a self-signed certificate
By requesting a certificate from Let's Encrypt
By creating an Azure App Service managed certificate

Obviously the third example can only work with Azure. The two others are written to work with Azure as well but can be adapted or used as an inspiration to work on other platforms.

Getting started

Set the variables of the root module

If you want to run this against your Azure infrastructure you will need to provide values for the variables of the root module:

The dns_zone_name should contain the name of your DNS Zone managed in Azure
The dns_zone_rg_name should contain the name of the resource group containing…

View on GitHub

To give some context very quickly, let's say we have used the app_service module of the GitHub repo to create an App Service bound to a custom domain, and now we need to secure this custom domain using a certificate.

We have already done this using a self-signed certificate, and with a Let's Encrypt certificate, now we are doing it using a managed one.

Level 3: let Azure handle the certificate stuff

The idea behind managed certificates is pretty simple: if you can prove you own a domain, then Azure will issue a certificate for you, valid for this domain.

And then you don't have to worry about it, Azure will renew the certificate for you, you can forget about it and focus on other things, like your business for instance...

On the diagram side, things looks way simpler than for the other "levels": The diagram looks almost the same as for self-signed certificates

On the code side, we have previously bound the App Service to a custom domain using a azurerm_app_service_custom_hostname_binding resource in the app_service module:

# Bind app service to custom domain
resource "azurerm_app_service_custom_hostname_binding" "app" {
  hostname            = "tf-certs-demo.${var.dns.zone_name}"
  app_service_name    = azurerm_app_service.app.name
  resource_group_name = azurerm_resource_group.rg.name

  depends_on = [azurerm_dns_txt_record.app]
}

In the managed module, creating the managed is done using the azurerm_app_service_managed_certificate with only the binding id as an argument:

resource "azurerm_app_service_managed_certificate" "managed" {
  custom_hostname_binding_id = var.custom_domain_binding_id
}

And finally, back in the main module, the azurerm_app_service_certificate_binding resource brings everything together by binding the managed certificate to the custom domain:

resource "azurerm_app_service_certificate_binding" "cert_binding" {
  certificate_id      = module.managed.certificate_id
  hostname_binding_id = module.app_service.custom_domain_binding_id
  ssl_state           = "SniEnabled"
}

Note that comparing to the other posts of the series, I have detailed a little bit more what's happening in the code before and after the generation of the certificate. Otherwise I would have only 3 lines of code to show 😬

The code snippets above are spread into 3 modules from the repo of the whole series.

I had previously made another repo dedicated to managed certificates using a single module if you prefer:

xaviermignot / app-service-managed-certificate-demo

Azure App Service Managed Certificate demo

This repository contains a simple demo of App Service managed certificates with Terraform.

View on GitHub

When browsing the App Service, we can see that the certificate is issued by GeoTrust (aka DigiCert) and of course trusted by the browser: The certificate is trusted and valid for 6 months ✅

Managed certificates: too good to be true ?

As you can see it's very simple to generate a managed certificate, and it's ~~free~~ included in the App Service Plan's pricing. It looks like the perfect solution for securing your Web Apps, but there is a downside that you should be aware of: if you want to use managed certificates, your App Service have to be publicly exposed.

So you can't use them if you put your Web Apps behind an appliance like an Application Gateway and enforce the traffic to go through this appliance.
Even if it's technically feasible to do so (by using tricky maneuvers, I have done this but I won't tell more 😅), this is not a target solution as it will prevent the certificate from being renewed by App Service.

There are other minor limitations like the lack of support of wildcard certificates and the impossibility to export the certificate but it seems fair to me and it did not bother me at all in my personal usage.

These limitations are listed here in the documentation, a little bit quietly so I think it's worth mentioning them.

To finish on a positive note, these limitations mean that managed certificate might not suit enterprise scenarios where we will probably put a security appliance in front of our App Services.

But for personal projects, or smaller ones, proof of concepts, or sandbox environments, it's really a great feature, as it makes really easy to expose you applications securely on a custom domain.

Managed certificates for other Azure services

Staying on a positive track before closing this post, I have to mention that managed certificates are not limited to App Service in Azure.

I have already used them for Azure CDN (in GA, supported by the AzureRm Terraform provider) and Azure API Management (currently in public preview, no support in Terraform yet).

It might also exist for other services so if your are about to use an Azure service associated with a custom domain, look for the support of managed certificates 😉

Wrapping up

As this post closes the series on certificate generation with Terraform for Azure, let's recap the whole series in a single table:

Generation method	Ease of use	Security level	Summary
Self-signed certificates	✅✅	🔓	You can start here but consider moving to the other method as soon as you can.
Let's Encrypt	✅	🔐🔐🔐	Free & trusted certs, require little maintenance once set-up. A little harder to understand at first but then can be used everywhere.
Managed certificates	✅✅✅	🔐🔐🔐	Use this unless your Web Apps can't be directly and publicly accessible. Or if you are not using Web Apps...

That's it for this post, I hope this series has been informative, as this is not the topic I am the most comfortable with (don't forget that I am a developer first, and most of us are scared by certificates 😱).

Anyway, there are several ways to automate certificate generation in Azure and in general, and the industry makes it easier that ever, through open initiatives like Let's Encrypt or public cloud specific features like managed certificates.

Choose the one that best suits your needs, and thanks for reading 🤓

TLS with Terraform and Azure: get certificates from Let's Encrypt

Xavier Mignot — Wed, 25 May 2022 19:46:42 +0000

Following my previous post on generating self-signed certificates with Terraform, this one is the second post of the series.

This time we are going to use Let's Encrypt as the certificate authority (CA) instead of our own machine. As a result we will get trusted certificates that can be used in production, for free.

Previously in the "TLS with Terraform and Azure" series...

If you haven't read my previous post you can check out this section to get the common context for the whole series.

There is also a GitHub repository that contains all the code from this series of posts:

xaviermignot / terraform-certificates

A repository showing how to generate certificates using Terraform

Certificate generation with Terraform for Azure App Service

This repository contains sample code to generate TLS certificates using Terraform.
It uses an Azure App Service as an example of a website to secure.

The certificates are generated in 3 ways:

By creating a self-signed certificate
By requesting a certificate from Let's Encrypt
By creating an Azure App Service managed certificate

Obviously the third example can only work with Azure. The two others are written to work with Azure as well but can be adapted or used as an inspiration to work on other platforms.

Getting started

Set the variables of the root module

If you want to run this against your Azure infrastructure you will need to provide values for the variables of the root module:

The dns_zone_name should contain the name of your DNS Zone managed in Azure
The dns_zone_rg_name should contain the name of the resource group containing…

View on GitHub

Level 2: requesting a certificate from Let's Encrypt

If you don't know what Let's Encrypt is, in a few words it's a free, automated and open certificate authority (CA), allowing everyone to get certificates trusted by browsers at no charge.

As the Let's Encrypt certificates are valid for 90 days (instead of one year for commercial authorities), automation is strongly recommended.

So how can we automate this with Terraform ? Using the third-party provider ACME. ACME stands for Automated Certificate Management Environment, the protocol used by Let's Encrypt. The Terraform ACME provider supports any ACME CA, so we need to configure Let's Encrypt's endpoint in the provider configuration:

terraform {
  required_providers {
    # The provider is declared here just like any provider...
    acme = {
      source  = "vancluever/acme"
      version = "~> 2.0"
    }
  }
}

# ...and is configured here, with the Let's Encrypt production endpoint.
provider "acme" {
  server_url = "https://acme-v02.api.letsencrypt.org/directory"
}

Let's Encrypts provides a staging endpoint you should use for testing: https://acme-staging-v02.api.letsencrypt.org/directory

There is a limit of 50 certificates per domain and per week on the production environment, on the staging it's 30,000 (but the certificates will not be trusted by the browser)

Once the provider properly configured, we can start creating resources. As for self-signed certificates, it starts here with a private key, and using this private key and an email we create a registration which is an account on the ACME server:

# Creates a private key in PEM format
resource "tls_private_key" "private_key" {
  algorithm = "RSA"
}

# Creates an account on the ACME server using the private key and an email
resource "acme_registration" "reg" {
  account_key_pem = tls_private_key.private_key.private_key_pem
  email_address   = var.email
}

Then we can request a certificate using our account, this is where the real magic happens:

# As the certificate will be generated in PFX a password is required
resource "random_password" "cert" {
  length  = 24
  special = true
}

# Gets a certificate from the ACME server
resource "acme_certificate" "cert" {
  account_key_pem          = acme_registration.reg.account_key_pem
  common_name              = var.common_name # The hostname goes here
  certificate_p12_password = random_password.cert.result

  dns_challenge {
    # Many providers are supported for the DNS challenge, we are using Azure DNS here
    provider = "azure"

    config = {
      # Some arguments are passed here but it's not enough to let the provider access the zone in Azure DNS.
      # Other arguments (tenant id, subscription id, and cient id/secret) must be set through environment variables.
      AZURE_RESOURCE_GROUP = var.dns.zone_rg_name
      AZURE_ZONE_NAME      = var.dns.zone_name
      AZURE_TTL            = 300
    }
  }
}

The key part is in the dns_challenge block of the acme_certificate resource.

Before generating a certificate for our domain, Let's Encrypt checks that we own that domain. To do that it proceeds with a DNS challenge, basically it generates a random string and will not generate the certificate unless that random string is in a specific TXT record of the DNS zone.

Obviously the ACME provider does that for us, we just need to let it access our DNS zone. The provider supports many DNS providers, in this case we are using Azure DNS.

So we need to tell the ACME provider where our DNS zone is, so we specify its name and resource group name in the config block above. Unfortunately the provider cannot use the Azure CLI authentication so we need to set additional arguments for authentication, so that the provider knows the subscription we are using, and a client id/secret to access it.

This is the kind of information that should not be pushed to a git repository, so I prefer to put these as environment variables in a .env git-ignored file like this:

export ARM_TENANT_ID="<YOUR AZURE TENAND ID (a guid)>"
export ARM_SUBSCRIPTION_ID="<YOUR AZURE SUBSCRIPTION ID (another guid)>"
export ARM_CLIENT_ID="<AN APP REGISTRATION ID (yep it's a guid too)>"
export ARM_CLIENT_SECRET="<THE APP REGISTRATION SECRET (not a guid this time)>"

Then I use the source .env command and the ACME provider can use the environment variables to authenticate to Azure and add/remove records in my DNS zone.

If you are using Terraform Cloud or running Terraform in a CI/CD context you will not need to do this as the environment variables are already set

Once the full code has been applied you can check out the Activity Log of your DNS zone.

You should see that the service principal corresponding to the environment variables has created and deleted a TXT record in the zone :
The creation and deletion of the TXT record should occur within a minute

And more importantly you can browse your App Service from its custom hostname and see how your browser enjoys this fresh trusted certificate:

Note the R3 issuer which is Let's Encrypt, and the absence of security warning ✅

Wrapping up

That's if for this post, which is the one why I have started this series. At first I had trouble understanding how this Let's Encrypt/ACME provider works and didn't find any content showing a full example.
So I hope this post makes sense and helps other to issue trusted certificates using Terraform.

Don't hesitate to dive into Let's Encrypt documentation as well, they explain in detail how the DNS challenge works, as well as the service itself and their certificate chain.

Back to the original diagram, here is where we are now:

The certificate is now issued by a trusted CA before being bounded to the App Service

The next post will close the series with Azure managed certificates, a feature that deserves more attention 😉

Thanks for reading !

TLS with Terraform and Azure: generate self-signed certificates

Xavier Mignot — Wed, 25 May 2022 19:39:14 +0000

We all love starting pet projects, so we tend to buy custom domains as it can be fairly cheap. SSL/TLS certificates on the other hand used to be pricey, but today there are several solutions to get these for free.

And this is for the good cause as every website should be secured by certificates nowadays.

This post is the first of a series where I will share 3 ways to automate the generation of certificates with Terraform for your Azure projects.

This one introduces the common workflow around an Azure Web App, and shows the first level of certificate generation using self-signed certificates.

The second post is using Let's Encrypt and the last one managed certificates.

Presentation of the context for the whole series

Let's say we have the following architecture as a starting point:

Let's start with a Web App bound to a custom domain

So we have the following components:

An App Service running in a plan with in the Basic tier at least
A DNS zone with at least the following records:
- A CNAME record pointing to the default App Service hostname (*.azurewebsites.net)
- A TXT records to verify the domain ownership
These two records allow the creation of a custom hostname binding at the App Service level

Everything is created using Terraform CLI from my terminal, including the DNS records, that's why I'm using Azure DNS 😎

With this setup the App Service is accessible on the custom hostname using HTTP only, if we try to access it using HTTPS our browser displays an error as the only certificate it can find is bound the azurewebsites.net domain:

The browser displays an error as the certificate doesn't match the hostname... yet

In the Azure portal, the custom domain is added to the App Service but marked as unsecured:
The Custom domains blade of the Azure portal displays this error on our custom domain

Now that we have our basic setup, let's see how we can secure this web app.

GitHub repository

All the code from this series of posts is available in the following repo:

xaviermignot / terraform-certificates

A repository showing how to generate certificates using Terraform

Certificate generation with Terraform for Azure App Service

This repository contains sample code to generate TLS certificates using Terraform.
It uses an Azure App Service as an example of a website to secure.

The certificates are generated in 3 ways:

By creating a self-signed certificate
By requesting a certificate from Let's Encrypt
By creating an Azure App Service managed certificate

Obviously the third example can only work with Azure. The two others are written to work with Azure as well but can be adapted or used as an inspiration to work on other platforms.

Getting started

Set the variables of the root module

If you want to run this against your Azure infrastructure you will need to provide values for the variables of the root module:

The dns_zone_name should contain the name of your DNS Zone managed in Azure
The dns_zone_rg_name should contain the name of the resource group containing…

View on GitHub

The readme explains how to get started if you want to create the resources and generate the certificates in your own subscription.

Level 1: generating a self-signed certificate

As a very first step we will generate a self-signed certificate, something we will not do for production but can be handy for a quick test.

HashiCorp provides a tls provider to generate the certificate using Terraform just like we will do using openssl in Linux or PowerShell in Windows.

Here is the HCL code to do this:

# Creates a private key in PEM format
resource "tls_private_key" "private_key" {
  algorithm = "RSA"
}

# Generates a TLS self-signed certificate using the private key
resource "tls_self_signed_cert" "self_signed_cert" {
  key_algorithm   = tls_private_key.private_key.algorithm
  private_key_pem = tls_private_key.private_key.private_key_pem

  validity_period_hours = 48

  subject {
    # The subject CN field here contains the hostname to secure
    common_name = var.common_name
  }

  allowed_uses = ["key_encipherment", "digital_signature", "server_auth"]
}

It's just two resources: a private key and a certificate generated using this private key.

We could stop here but in an Microsoft context we need to export the certificate in PFX format to use it in an Azure service such as App Service or Application Gateway.

This is done by combining the use of the HashiCorp random provider (to generate a password) and the third-party provider pkcs12:

# To convert the PEM certificate in PFX we need a password
resource "random_password" "self_signed_cert" {
  length  = 24
  special = true
}

# This resource converts the PEM certicate in PFX
resource "pkcs12_from_pem" "self_signed_cert" {
  cert_pem        = tls_self_signed_cert.self_signed_cert.cert_pem
  private_key_pem = tls_private_key.private_key.private_key_pem
  password        = random_password.self_signed_cert.result
}

# Finally we push the PFX certificate in the Azure webspace
resource "azurerm_app_service_certificate" "self_signed_cert" {
  name                = "self-signed"
  resource_group_name = var.resource_group_name
  location            = var.location

  pfx_blob = pkcs12_from_pem.self_signed_cert.result
  password = pkcs12_from_pem.self_signed_cert.password
}

And that's it, the certificate is ready to be used as an App Service certificate as you can see in the full code here.

As you have probably guessed this certificate will not make the browser happy as our machine is not recognized as a trusted authority:

The hostname matches but the browser still displays an error as the authority is not trusted

But at least the Azure portal is happier as it considers that the custom domain is now secured:
The Azure portal seems less picky than the browser...

Wrapping up

That's it for this first step, which serves as an introduction to next posts. As I already mentioned the use of self-signed certificates is not recommended for production, and next you will see we can do much better than that.

If we take a look back to our initial diagram, we have made the following changes:
Diagram of the generation and deployment of the self-signed cert

Stay tuned for the next post who will be about Let's Encrypt. If you already need it, you can go to my repo as the code is already there 🤓

Thanks for reading !

Use Terraform Cloud for your pet projects

Xavier Mignot — Wed, 25 May 2022 19:26:26 +0000

Over the last few years I have been interested by IaC (Infrastructure as Code), and started recently to use Terraform in my professional life. To keep practicing my Terraform skills I also started to use it for POCs and personal projects instead of using the portal.

Even if I had to push myself at the beginning, I now have a routine to initialize and manage my Azure resources that I will share in this post. It relies on Terraform Cloud that you can use for free !

What is Terraform and Terraform Cloud ?

This post is not a deep dive into Terraform, we will stick to the basics. But if you are very new to Terraform, I encourage you to browse the product website and follow a basic tutorial as it's the best way to know what the tool is about.

To better understand the differences between Terraform and Terraform Cloud, let's sum up the differences real quick:

Terraform is the open-source CLI tool with the plan, apply, destroy commands that can run anywhere (including on your machine or in Terraform Cloud)
Terraform Cloud is a SaaS offering to manage Terraform runs and states remotely

Is it free ?

Terraform CLI is open-source and free to use. Terraform Cloud is also free for teams up to 5 users, so it's perfect for pet projects.

Terraform Cloud has paid plans for larger organizations, and also a self-hosted version called Terraform Enterprise. That's not the topic of this post, but it's important to know how the company behind the tool makes money.

Create your Terraform Cloud workspace

Let's get finally to the point, shall we ? For this sample we will create an Azure Function App which will result in creating the following resources:

A resource group
An App Service Plan
A storage account
A Function App
An Application Insights account (optional but always handy to monitor the app's telemetry)

Prerequisites

To do this we will need the following prerequisites:

A Terraform Cloud account with an organization, to create one follow the guidelines from here
An Azure subscription, a free one will be enough
Azure CLI installed and connected to your subscription

Create your workspace

Once you have your organization (which represents your team) in Terraform Cloud, you will need a workspace (which represents your project).

Go to Terraform Cloud, select you organization, and click on the "New workspace" button.

On the next page you have to choose a workflow, choose CLI-driven workflow:

The CLI-driven workflow on the new workspace page

Using this workflow allows you to run Terraform commands from your machine, but they will run remotely in Terraform Cloud. It's the best of two worlds approach as you get the simplicity of running the commands from your terminal of choice, and you get the security of having the state stored remotely, and the runs are accessible in Terraform Cloud.

Give Terraform Cloud access to your Azure subscription

Next you need to connect your workspace with you Azure subscription. Start by creating a new service principal with this Azure CLI command:

$ az ad sp create-for-rbac -n '<SERVICE PRINCIPAL NAME>' --role Owner

Choosing a relevant name will be more helpful than the default azure-cli-datetime name. Specifying the Owner role is needed if you plan to assign a role to a resource, using a Managed Identity for instance.

The command will output a JSON object:

{
  "appId": "<A NEW GUID>",
  "displayName": "service-principal-name",
  "name": "http://service-principal-name",
  "password": "<A SUPER SECRET STRING>",
  "tenant": "<YOU TENANT ID>"
}

Then back in Terraform Cloud you need to create environment variables from the Variables tab of your workspace page:

ARM_CLIENT_ID: The Service Principal's id if from the previous command's output
ARM_CLIENT_SECRET: The Service Principal's secret also from the command output (you should tick the Sensitive checkbox for this one)
ARM_SUBSCRIPTION_ID: Your subscription id, you can get it with this command: az account show --query id -o tsv
ARM_TENANT_ID: Your tenant id, also from the command output

You should get something like this:
These variables are then used by Terraform to get tokens for calling the Azure REST API.

Set a few variables for your project

Still from the Variables tab of your workspace page, you need to create the following Terraform variables:

project contains a string used in the names of the Azure resources. It has to be something that will make the names unique globally
location contains the name of the Azure region you want to use, in az cli style such as eastus, westus, westeurope, etc.

Note that these variables are not environment variables, they are not used in the shell but in the Terraform code.

Here is how I have set the variables in my workspace:
My resources will be created in the France Central Azure region, and called afa-xmi-tf-sample, ai-xmi-tf-sample, etc.

Get the code & get ready to deploy

I have made up a GitHub repo with the resources listed above, you can grab it here.

So you don't have to write the Terraform code for now but you can explore it and tweak it as you want.

Repository content

For simplicity I have put all the files at the root of the repository. In a "full" project I would have several folders dedicated to the Terraform files, the source, the tests, the doc, etc.

But there is only Terraform files here so it's pretty simple, we can just note that we have:

az-*.tf files containing the creation of the Azure resources
tf-*.ft files containing Terraform configuration such as the variables declaration and the required providers

Add the `tf-backend.tf` file

There is one file that you need to add to establish the link between the repo and your Terraform Cloud workspace. Just create the tf-backend.tf file with the following structure:

terraform {
  backend "remote" {
    organization = "<YOUR ORGANIZATION NAME>"

    workspaces {
      name = "<YOUR WORKSPACE NAME>"
    }
  }
}

This file is git-ignored as it contains the name of the Terraform Cloud organization and workspace. Even if those are not secrets, I consider as good practice not to commit environment-related values like this, especially in public repositories.

Let's deploy the resources !

We are finally getting to the point where we are going to deploy some stuff in Azure. Let's jump in your favorite terminal in the root of the repo if you are not already there, and fire a few commands.

First and just once run:

terraform login to authenticate to your Terraform Cloud workspace using your browser
terraform init to install the required providers

Then you are ready to use the most common Terraform CLI commands:

terraform plan to create a plan to let Terraform determine the changes to make on your infrastructure
terraform apply to apply the plan and finally make the changes (which will result in the creation of all resources when you run this for the first time)

After that feel free to tweak the Terraform files, experiment, and run the plan and apply commands again. Use the Azure portal to see what you have deployed:
You should get something similar to this

Once you are done, you can delete all the resources with the terraform destroy command.

What I like about this approach is that even if you can see the output of the commands in you terminal, everything is running in the cloud, and the state of the infrastructure is securely stored in the cloud as well.

Wrapping up & next steps

We have seen in this post how to deploy resources using the CLI approach of Terraform Cloud. While it might seem overkill to do this for personal projects or POCs, I think it's a good compromise as it combines the good practice to have the state securely stored in the cloud with the comfort of running the commands from my terminal (and not to have to push a change to trigger a CI/CD pipeline).

When I finish late my work on a project, I also like to run terraform destroy, approve the changes and close my laptop right away, as everything is running somewhere else... in the cloud !

This post describes a first step in my Terraform journey, and a big one as I needed quite some time to make up this post 😇

Then I might go further with the following next steps:

Automate things: I could write a script to create the workspace, the service principal and configure everything in a single step
Manage environments: while it's not necessary in the early stages of pet projects, being able to manage several environments (DEV, PROD, ...) could be interesting to do, and useful for my professional life
Improve security: creating a service principal with the Owner role does not following the principle of least privilege... there must be a way to ensure that its access are scoped to the dedicated resource group
Use Bicep: not related to Terraform at all but I know I will try this new way of deploying stuff in Azure some day 😅

That's it for this post, thanks for reading, don't hesitate to reach out, and happy terraforming 🤓

The Ops Community ⚙️: Xavier Mignot

Azure App Service, deployment slots (with configuration !) and Terraform

The problem with deployment slots and Terraform state

TL;DR: The solution, "briefly" explained

GitHub repository

xaviermignot / terraform-app-service-slots

A demo for using Azure App Service deployment slots using Terraform

Azure App Service with Terraform, deployment slots and app settings

Getting started

The demo, in action

Provision resources, and deploy code in both slots

A little glimpse inside the IaC

Let's swap !

In case of a rollback...

What about other changes to the infrastructure ?

A few words about the azurerm_web_app_active_slot resource

Wrapping up

Deploy Azure Logic Apps as code

Presentation of the context

GitHub repository

Creating the base resources

Creating the API Connection

Creating the Logic App workflow

Why are Logic Apps different ?

Let's develop/deploy this Logic App for good now !

What about Terraform ?

Wrapping up

TLS with Terraform and Azure: use managed certificates

Previously in the "TLS with Terraform and Azure" series...

xaviermignot / terraform-certificates

A repository showing how to generate certificates using Terraform

Certificate generation with Terraform for Azure App Service

Getting started

Set the variables of the root module

Level 3: let Azure handle the certificate stuff

xaviermignot / app-service-managed-certificate-demo

Azure App Service Managed Certificate demo

Managed certificates: too good to be true ?

Managed certificates for other Azure services

Wrapping up

TLS with Terraform and Azure: get certificates from Let's Encrypt

Previously in the "TLS with Terraform and Azure" series...

xaviermignot / terraform-certificates

A repository showing how to generate certificates using Terraform

Certificate generation with Terraform for Azure App Service

Getting started

Set the variables of the root module

Level 2: requesting a certificate from Let's Encrypt

Wrapping up

TLS with Terraform and Azure: generate self-signed certificates

Presentation of the context for the whole series

GitHub repository

xaviermignot / terraform-certificates

A repository showing how to generate certificates using Terraform

Certificate generation with Terraform for Azure App Service

Getting started

Set the variables of the root module

Level 1: generating a self-signed certificate

Wrapping up

Use Terraform Cloud for your pet projects

What is Terraform and Terraform Cloud ?

Is it free ?

Create your Terraform Cloud workspace

Prerequisites

Create your workspace

Give Terraform Cloud access to your Azure subscription

Set a few variables for your project

Get the code & get ready to deploy

Repository content

Add the tf-backend.tf file

Let's deploy the resources !

Wrapping up & next steps

A few words about the `azurerm_web_app_active_slot` resource

Add the `tf-backend.tf` file