Intro

A few days ago I needed to integrate a service with Terraform in a way that allows my team to create infrastructure (with Terraform) while storing some information of that infrastructure in a remote service which doesn't have a Terraform provider, I also wanted to include the possibility to extract some information from this remote service to use in the resources managed by Terraform.

It is not important which service I was working with but the fact that you can interact with any external service that exposes an API or that you have access to query (and write) to it.

By the end of this tutorial you will be able to write a Terraform module that will allow you to read and write to an external service that doesn't have a terraform provider.

Initial investigation

While thinking on how to integrate this provider-less service I found that Terraform does have a way to interact with external services, it is by no means the same as a fully Terraform provider, the main difference is that with a provider you can manage the complete lifecycle of the resources managed with Terraform very easily while with this approach I will explain it is a bit limited, functional but limited.

We will be using two scenarios in this tutorial, how to read information from an external resource and how to write to that external resource. Both approaches use a different implementation that we will be covering here.

Let's read!

For this tutorial I will use the pokemon api as an external resource to retrieve the name of the first ability of a pokemon and use it as name of my terraform resource.

The first thing we need to know is that Terraform has something called External Data Source which allows you to execute an script and send back to terraform the output of this script, we will need to write an script that connects to the API and returns something that out data source will expose later, here is an example:

/module/poke.ps1

$stdio = [Console]::In.ReadLine()
$stdio = ConvertFrom-Json $stdio

$url = "https://pokeapi.co/api/v2/pokemon/$($stdio.name)"
$response = Invoke-RestMethod "$url" -Method Get;
$ability_name = $response.abilities[0].ability.name

Write-Output "{""value"" : ""$ability_name""}"

This simple script will read from the stdio and parse it as a Powershell object to extract the name of the pokemon and then access our external API, then will retrieve the name of the first ability and write a json object to stdio containing this value.

This is basically what the terraform documentation details on how this works. Let's write now a terraform module to use this:
/module/poke.tf

variable "name" {
  type        = string
  default     = ""
  description = "Name of the pokemon"
}

data "external" "read_ability" {
  program = ["powershell.exe", "./main.ps1"]
  query = {
    name    = var.name
  }
}
output "content" {
  value = data.external.read_ability.result.value
}

Whit this terraform code we can pack this .tf and the .ps1 files into a single folder and use them as a module from other .tf files as follow:
/main.tf

module "read_ability" {
  source = "./module"
  name = "ditto"
}

resource "azurerm_resource_group" "example" {
  name     = module.read_ability.content
  location = "West Europe"
}

keep in mind the folder structure and where each file needs to be to make this work.

When you run a Terraform plan on the main.tf file you will see the name of the resource group will be limber, in this way you can pass other names to the poke module and obtain different names, of course this is a very limited example just to demonstrate how this works but you are encouraged to do your own implementation with more functionality.

Let's write

Now that we can read from our external service, we can create resources in Terraform using that information as input, but what if we want to store something we created with terraform in an external service. Let's suppose we have an external store and we want to keep track of the ID of some resources created in Azure in that store. For this example we will be using kind of the same approach as for read , this means we will create a terraform module that will take an input and write it to some external service.

/module/write.ps1

param (
  $Value
)

# Implement here your code to write to the external service
# here we only do a write-host to demonstrate the functionality
Write-host $Value

/module/main.tf

variable "name" {
  type        = string
  default     = ""
}

resource "null_resource" "write_to_external_service" {
  triggers = {
    "command" = "${timestamp()}"
  }
  provisioner "local-exec" {
    command     = "powershell -file write.ps1 -Value ${var.name} "
    working_dir = "${path.module}"
  }
}

main.tf

resource "azurerm_resource_group" "example" {
  name     = module.read_ability.content
  location = "West Europe"
}

module "write_value" {
  source = "./module"
  name = resource.azurerm_resource_group.example.name
}

The /module/main.tf is using a null_resource to execute a command and passing a single argument to the script, in this case "name" and the trigger will force to run every time this null resource is called, be aware you need to implement in your script a mechanism to not write multiple times the same outputs to your external service, but that is up to you and the way your external service works.

Wrapping up

So in this short tutorial you ended up with two modules, one to read from the PokeAPI and another to write to an external service, in both cases using a Powershell script to interact with the external services, this is an easy way to integrate a service without a terraform provider into a terraform deployment without the extra effort that means build a fully functional terraform provider (which of course is the prefered way to deal with external sources).

As a workaround to quickly get it to work, this is a very good alternative that Hashicorp provides us, I hope this helps you and in case you find this useful, please consider to share it with others so we all benefit from others contributions!.

Follow me in my other networks and get in touch if you have any question or need help 👍