Terraform: count, for_each, and for loops

Using for, count, and for_each loops in Terraform with examples

In the previous post, we got an overview of the Terraform data types — Terraform: introduction to data types — primitives and complex.

Now let’s see how these types can be used in loops.

Terraform supports three types of cycles:

• count: the simplest, used with a given number or with the length() function; uses indexes from a list or map for iteration
• suitable for creating identical resources that will not change
• for_each: has more options, used with a map or set, uses sequence key names to iterate
• suitable for creating resources of the same type, but with the ability to set different parameters
• for: used to filter and transform objects with lists, sets, tuples or maps; can be used in conjunction with operators and functions like if, join, replace, lower, or upper

Terraform count

So, count is the most basic and first method for performing tasks in a loop.

Takes either a number, list or a map as an argument, performs iteration, and assigns an index to each object according to its position in the sequence.

For example, we can create three AWS S3 buckets like this:

resource "aws_s3_bucket" "bucket" {
  count = 3

  bucket = "bucket-${count.index}"
}

As a result, Terraform will create an array of three buckets named bucket-0, bucket-1, and bucket-2.

We can also pass in alist and use the length() function to get a number of elements in that list, and then iterate through each of them using their indexes:

variable "projects" {
  type = list(string)
  default = ["test-project-1", "test-project-2", "test-project-3"]
}

resource "aws_s3_bucket" "bucket" {
  count = length(var.projects)

  bucket = "bucket-${var.projects[count.index]}"
}

In this case, three buckets will be created with the names “bucket-test-project-1”, “bucket-test-project-2”, and “bucket-test-project-3”.

To get the value of the names of the buckets that were created in this way, we can use “*” to select all indexes from the aws_s3_bucket.bucket array:

...
output "bucket_names" {
  value = aws_s3_bucket.bucket[*].id 
}

But there is one important nuance with the count: because of the binding of elements to indexes, you can get an unexpected result.

For example, if you create these three buckets, and then add a new project at the beginning or in the middle of the list, Terraform will remove the buckets for the projects after the added one, because the object indexes will change in the list.

That is:

variable "projects" {
  type = list(string)
  default = ["test-project-1", "another-test-project", "test-project-2", "test-project-3"]
}

This will lead to the:

$ terraform apply
...
  # aws_s3_bucket.bucket[1] must be replaced
-/+ resource "aws_s3_bucket" "bucket" {
...
      ~ bucket = "bucket-test-project-2" -> "bucket-another-test-project" # forces replacement
...
  # aws_s3_bucket.bucket[2] must be replaced
-/+ resource "aws_s3_bucket" "bucket" {
...
      ~ bucket = "bucket-test-project-3" -> "bucket-test-project-2" # forces replacement
...
  # aws_s3_bucket.bucket[3] will be created
  + resource "aws_s3_bucket" "bucket" {
...
      + bucket = "bucket-test-project-3"
...
Plan: 3 to add, 0 to change, 2 to destroy.

And if there is data in the buckets, the deployment will stop with the BucketNotEmpty error because Terraform will try to delete the buckets.

However, count is great if you need to check a condition like “create a resource or not”. This can be done as follows:

variable "enabled" {
  type = bool
  default = true
}

resource "aws_s3_bucket" "bucket" {
  count = var.enabled ? 1 : 0

  bucket = "bucket-test"
}

That is, if enabled = true then we create 1 bucket, if false – then 0.

Terraform for_each

The for_each loop allows you to perform iterations more flexibly.

It accepts a map or set, and for iteration uses each key and value instead of indexes. In this case, a number of keys will determine the number of resources that will be created.

Because each one key is unique, changing the values ​​in a set/map does not affect how resources are created.

In addition to the set and map you can use the list type, but it will have to be “wrapped” in a function toset() to turn it into set, from which for_each can get a key:value pair. In this case, the key value will be == the value value.

for_each with set and list

So, if we take the same resource aws_s3_bucket, using for_each we can create buckets like this:

variable "projects" {
  type = set(string)
  default = ["test-project-1", "test-project-2", "test-project-3"]
}

resource "aws_s3_bucket" "bucket" {
  for_each = var.projects

  bucket = "bucket-${each.value}"
}

Or with a variable with the list type and toset() function for the for_each:

variable "projects" {
  type = list(string)
  default = ["test-project-1", "test-project-2", "test-project-3"]
}

resource "aws_s3_bucket" "bucket" {
  for_each = toset(var.projects)

  bucket = "bucket-${each.value}"
}

But as a result, we will get not an array of data, but a map with individual objects:

...
  # aws_s3_bucket.bucket["test-project-1"] will be created
...

And then we will not be able to simply call aws_s3_bucket.bucket[*].id in the outputs.

Instead, we can use the values() function to retrieve all resources from the aws_s3_bucket.bucket:

...
output "bucket_names" {
  value = values(aws_s3_bucket.bucket)[*].id 
}

for_each with map

Another example with a map to create a tag Name:

variable "projects" {
  type = map(string)
  default = {
    "test-project-1" = "Test Project 1",
    "test-project-2" = "Test Project 2",
    "test-project-3" = "Test Project 3",
  }
}

resource "aws_s3_bucket" "bucket" {
  for_each = var.projects

  bucket = "bucket-${each.key}"
  tags = {
    "Name" = each.value
  }
}

Or using the merge() function to add common tags + tag Name (see also default_tags):

variable "projects" {
  type = map(string)
  default = {
    "test-project-1" = "Test Project 1",
    "test-project-2" = "Test Project 2",
    "test-project-3" = "Test Project 3",
  }
}

variable "common_tags" {
  type = map(string)
  default = {
    "Team" = "devops",
    "CreatedBy" = "terraform"
  }
}

resource "aws_s3_bucket" "bucket" {
  for_each = var.projects

  bucket = "bucket-${each.key}"
  tags = merge(var.common_tags, {Name = each.value})
}

As a result, we will get three tags:

...
  ~ resource "aws_s3_bucket" "bucket" {
        id = "bucket-test-project-1"
      ~ tags = {
          + "CreatedBy" = "terraform"
          + "Name" = "Test Project 1"
          + "Team" = "devops"
        }
...

for_each with a map of maps and attributes

Or you can even use a map of maps, and for each bucket pass a set of parameters, and then access a parameter via each.value.PARAM_NAME.

For example, in one parameter we can set the Name tag, and in another – a object_lock_enabled parameter:

variable "projects" {
  type = map(map(string))
  default = {
    "test-project-1" = {
      tag_name = "Test Project 1", object_lock_enabled = true 
    },
    "test-project-2" = {
      tag_name = "Test Project 2", object_lock_enabled = false
    },
    "test-project-3" = {
      tag_name = "Test Project 3", object_lock_enabled = false
    }
  }
}

variable "common_tags" {
  type = map(string)
  default = {
    "Team" = "devops",
    "CreatedBy" = "terraform"
  }
}

resource "aws_s3_bucket" "bucket" {
  for_each = var.projects

  bucket = "bucket-${each.key}"
  object_lock_enabled = each.value.object_lock_enabled
  tags = merge(var.common_tags, {Name = each.value.tag_name})
}

The result:

Terraform for

Unlike count and for_each, the for method is not used to create resources but for filtering and transformation operations on variable values.

The syntax for the for looks like this:

[for <ITEM> in <LIST> : <OUTPUT>]

Here an ITEM is the name of a local to the loop variable, a LIST is the list in which the iteration will be performed, and an OUTPUT is the result of the transformation.

For example, we can output bucket names as UPPERCASE as follows:

...
output "bucket_names" {
  value = [for a in values(aws_s3_bucket.bucket)[*].id : upper(a)]
}

for and conditionals expressions

We can also add a filter before the OUTPUT, i.e. perform an action only on some objects from the list, for example:

output "bucket_names" {
  value = [for a in values(aws_s3_bucket.bucket)[*].id : upper(a) if can(regex(".*-1", a))]
}

Here we are using can() and regex() functions to check the value of the a variable, and if it ends with the “-1”, then we perform upper(a):

...
bucket_names = [
  "BUCKET-TEST-PROJECT-1",
]

for and iteration over a map

It is possible to iterate over a key:value pair from a map variable:

variable "common_tags" {
  type = map(string)
  default = {
    "Team" = "devops",
    "CreatedBy" = "terraform"
  }
}

output "common_tags" {
  value = [for a, b in var.common_tags : "Key: ${a} value: ${b}"]
}

As a result, we will get an object of the list type with values:

...
common_tags = [
  "Key: CreatedBy; Value: terraform;",
  "Key: Team; Value: devops;",
]

With the help of the => operator, we can convert a list to a map:

output "common_tags" {
  value = { for a, b in var.common_tags : upper(a) => b }
}

The result:

...
common_tags = {
  "CREATEDBY" = "terraform"
  "TEAM" = "devops"
}

for and for_each to iterate over complex objects

You can make a single variable that will have different data types for different values ​​and then iterate over it using for_each and for together.

For example, let’s create a variable of the list type, which will contain values ​​of the object type, and in the object we will set two fields – one of the string type, and one for a list of tags of the list type:

variable "projects" {
  type = list(object({
      name = string
      object_lock_enabled = string
      tags = map(string)
  }))

  default = [
    {
      name = "test-project-1"
      object_lock_enabled = "true"
      tags = {
          "Name" = "Test Project 1"
          "Team" = "devops"
          "CreatedBy" = "terraform"          
        }
    },
    {
      name = "test-project-2",
      object_lock_enabled = true,
      tags = {
          "Name" = "Test Project 2",
          "Team" = "devops",
          "CreatedBy" = "terraform"          
        }
    },
    {
      name = "test-project-3",
      object_lock_enabled = true,
      tags = {
          "Name" = "Test Project 3",
          "Team" = "devops",
          "CreatedBy" = "terraform"          
        }

    }        
  ]
}

resource "aws_s3_bucket" "bucket" {
  for_each = { for a in var.projects : a.name => a }

  bucket = "bucket-${each.key}"
  object_lock_enabled = each.value.object_lock_enabled
  tags = { for key,value in each.value.tags : key => value }
}

Then, in the aws_s3_bucket resource, to the loop we can pass the var.projects.name value, and for the tags, we loop through each resource from the list, and in each resource, we create a key:value with the each.value.tags.

for and String Directives

Documentation — Strings and Templates.

The syntax for iteration over a map will be as follows:

%{ for <KEY>, <VALE> in <COLLECTION> }<RESULTED_TEXT>%{ endfor }

That is, we can create a text file with the content of the variable values:

resource "local_file" "foo" {
  content = "%{ for a, b in var.common_tags }Key: ${a}\nValue: ${b}\n%{ endfor }"
  filename = "foo.txt"
}

The result:

$ cat foo.txt 
Key: CreatedBy
Value: terraform
Key: Team
Value: devops

Done.

Originally published at RTFM: Linux, DevOps, and system administration.