Infrastructure as Code (IaC) has become the default method for managing cloud resources in modern cloud-native teams.
The Global IaC tools market is also on a growth trend and is expected to be worth $2.8 Billion by 2028. Two main factors lead to the rise in the popularity of these tools. First, the increasing complexity of cloud architecture makes managing these resources manually repetitive, time-consuming, laborious, and also prone to human errors. It is also cost-inefficient as manual changes must be replicated multiple times and is not a productive use of your engineering team’s time. These together make IaC viable, and its adoption is already high among larger organizations.
This blog explores how IaC makes cloud resource management more reliable and simplified. We further list the top most popular vendor-neutral IaC tools and compare offerings by the major cloud providers.
Infrastructure as Code is the method of provisioning and managing resources (infrastructure) declaratively, in source control, just as one handles code. Some of the tools allow resources beyond just cloud infrastructure to be defined, such as applications, configurations, etc. In addition to deploying your application’s infra components, you are also able to deploy the associated network, monitoring, security, and observability tools using IaC instructions.
The IaC definition files are often stored along with the application code in the version control systems.
Here are a few key concepts in IaC.
IaC, being declarative in nature, guarantees the final state of the target system is exactly as defined in the input configuration. This is hugely beneficial since the users can just focus on specifying the desired end-state instead of describing a procedure imperatively to get to that end-state. This is essential for managing multiple resources, replicated across environments and regions, at scale. Two typical approaches to defining IaC are Declarative IaC and Procedural IaC. The Declarative definition instructs how the system needs to look once the configuration process is complete, i.e., the end result. In contrast, procedural/imperative definitions are step-by-step instructions on what the system needs to do to reach the desired configuration.
The IaC definition needs to be run to affect the target systems and shift them to the desired state. This is achieved by either pulling the configurations by a control plane or pushing the changes to a repository whenever there is a change in the configuration - push vs pull methods.
Based on your use case, you might provision mutable or immutable infrastructure.
Mutable infrastructure brings more flexibility as it can be modified or updated after it is provisioned. It allows teams to make ad hoc customizations, patch security issues, and update application requirements. While there is ease due to the flexibility offered, it is also important to have a system to document the changes. While mutable infra is sometimes needed, it comes at the expense of potential deviation of desired config from actual state (also called drift)
Immutable infrastructure, once provisioned, cannot be modified. If any change is required in immutable infrastructure, then the existing instance must be destroyed and a new instance must be provisioned. This tends to be a more reliable system.
The primary drivers for IaC adoption have been the increase in the number of deployments, the rising complexity of cloud services and architecture, and the need for cloud systems to scale up and down according to the load. Let us go through the benefits it achieves one by one.
Repeatability of deployments - Teams have gone from a few deployments every month to hundreds of deployments every day. At this pace, it is crucial to have a reliable and automated infrastructure management system. IaC provides a stable, tested, and collaborative framework for deploying and managing infra at scale and at pace.
The complexity of the cloud - All major cloud providers have between 150-200 tools and services they offer. This, in addition to your complex hybrid cloud architectures, requires a global solution that can be quickly deployed and also adheres to your security standards.
Scaling requirement - Once you have your infra requirements defined as code, it is easier to scale up and down with minimal time and cost investment.
Version Control - Like any code, IaC can be version controlled, so in the event of distress or anomaly in your systems, it can be rolled back and investigated.
Declarative paradigm - With IaC, provisioning infrastructure has been immensely simplified. There is no need to go through thousands of documentation pages and constantly fiddle with the state of your infra. With the declarative paradigm, you set the desired state and the controller provisions and maintain the system configuration at that state.
Ability to collaborate - Since IaC is handled just like code, it provides enhanced collaboration opportunities. It can be in the form of collaboration through version control systems or through Cloud Engineering platforms like Pulumi.
Automation of processes - The technical skills and budgets required to manage complex cloud environments can burden your company’s budget. With IaC, just a few engineers can easily manage your cloud infra.
Best practices - IaC provides the ability to meet high availability requirements, security standards, and reduce the risk of operating the system. Changing and maintaining the configurations to meet your customer’s needs becomes as easy as updating a single value in the config file.
Several different tasks are involved in setting up and maintaining your app and infra. Many different IaC tools exist today to help solve these tasks in their own way. Key tasks include provisioning, deployment, configuration, and orchestration.
Some tools are involved in the infrastructure setup, while others manage the infra or the applications in your environment. Below we look at some popular tools, both cloud service providers and vendor-neutral solutions.
Terraform by Hashicorp is the most popular IaC tool in use today. It comes in open-source self-managed, and Terraform Cloud (managed) options. Its ability to define cloud and on-prem resources in human-readable format make it highly capable. It is also cloud platform-agnostic, and the modules that you create can also have versions, be reused, and be shared.
Terraform’s plugins interact directly with the cloud provider, SaaS provider, and other APIs. A list of publicly available Terraform providers can be found in the Terraform Registry. You can directly use one of the publicly available modules from the registry or write your own Terraform module.
Terraform does more than just infrastructure configuration management. It enables initializing your infrastructure from scratch. And also lets you define network resources and combine resources from different providers.
The configuration language is declarative. And providers automatically calculate dependencies between resources so that they are created and deleted in the correct order. It also seamlessly integrates with VCS like GitHub so that you can make infrastructure changes with a simple merge.
Crossplane is a framework for building cloud-native control planes without the need to write code. It is built specifically for Kubernetes with support for multi-cloud, serverless, and containerized workloads.
Crossplane provides building blocks that enable you to provision, compose, and consume infrastructure with the Kubernetes API. These individual concepts work together to allow for a powerful separation so that each member of a team interacts with Crossplane at an appropriate level of abstraction.
For enterprises looking for more stability, better support, and reduced risk, there is the Universal Crossplane (UXP) product by Upbound.
Founded in 2017 by Microsoft and Google veterans, Pulumi is a free, open-source infrastructure as code tool. The Pulumi Service offering makes managing infrastructure secure, reliable, and hassle-free. Pulumi service is available as both a self-host or SaaS option. You can easily deliver IaC through your existing CI/CD pipelines. There is also an option to enforce guardrails for security and compliance using policies in standard languages.
You can view the roadmap and contribute to the Pulumi open-source project here. One of their main reasons for success has been their ability to support and help teams migrate to modern containerized, serverless, and Kubernetes workloads easily. Find a detailed comparison between Pulumi and Terraform here.
Image source: https://www.pulumi.com/product/#deploy
AWS CloudFormation lets you model, provision, and manage AWS and third-party resources from scratch by treating your infrastructure as code. The AWS CloudFormation registry allows you to model and provision third-party resources and modules published by the AWS partner network and developer community.
The CloudFormation templates are stored as text files that comply with open-source declarative languages JSON and YAML standards. There is also the AWS CloudFormation designer if you prefer to design visually. Start by checking out the Sample templates as they demonstrate how you can create templates for various use cases.
A stack implements and manages the group of resources outlined in your template, and allows the state and dependencies of those resources to be managed together. A change set is a preview of changes that stack operations will execute to create, update, or remove resources.
Multiple templates can also be pooled together as StackSets. Using this, you can perform multiple operations (Create, Update, Delete), across multiple AWS regions and multiple accounts in one go.
The best part is that you can get started for free - 1,000 handler operations per month per account are covered in the AWS Free tier.
Azure resource manager is the deployment and management service for Azure. It acts as a management layer that receives requests from your Azure APIs, CLI, or SDKs. Then, it authenticates and authorizes the request before forwarding it to the respective Azure service. Operations include creating, updating, and deleting Azure resources, and the management features like access control, locks, logs, and tags help you organize your resources after deployment.
To implement IaC for your Azure cloud resources, you use Azure Research Manager (ARM) templates. The IaC for Azure is written in Bicep, a domain-specific language that uses declarative syntax. The Bicep code is converted into JSON files during deployment by Bicep CLI. ARM templates can also be written directly in JSON format.
Google’s version of an IaC for cloud resources is the Google Cloud Deployment Manager. Using this, you can easily create a set of Google Cloud resources and manage them as a unit called deployment.
The infrastructure type, structure, and property are defined in the configuration file. The configuration is defined in YAML format, templates are in Python or Jinja2 formats. You can further break your configuration into templates, which are separate files that can be imported as a type in configuration. You can use as many types as you like in a configuration.
A list of all the supported resource types can be found here. Deployment manager limits and quotas can be found here. Each deployment has a predefined set of environment variables set by the Deployment Manager. You can use these variables in your templates to get information about the deployment/project.
Vendor-neutral and CSP tools are the two main types of cloud IaC tools. Cloud provider tools are very similar in that they are primarily built to provision the infrastructure of their respective cloud.
However, AWS CloudFormation has higher adoption due to it being released earlier than the others. AWS Cloud formation configurations are coded in JSON and YAML formats. AWS CloudFormation also supports many third-party provider resources in its registry. It also comes with 1,000 handler operations per month with the free tier.
The configuration language is one thing that is different among these. Azure Resource Manager uses Bicep language and translates it into the ARM JSON template during deployment. Google’s deployment manager templates can be configured in Python, Jinja 2, and YAML.
The more important comparison here is between the vendor-neutral tools. These provide higher versatility, additional features, and a free version to help anyone get started.
|Configuration Language||HCL, JSON||JS, Typescript, Python, Go, .NET, YAML||YAML|
|Supported resources||1000s across AWS, GCP, Azure, and more||~120 provider packages||~70 from Kubernetes providers|
|Deploy to multiple locations||Yes||Yes||Yes|
|Primary function||Provisioning||Cloud Engineering||Control plane|
|Paradigm||Declarative API wrapper. Programmatic with CDK||Programmatic, declarative||Declarative with continuous reconciliation|
|Pricing||Free, with paid managed offering||Free, with paid managed offering||Free, with paid managed offering|
|Target user||DevOps Engineer||Platform Engineer||DevOps Engineer|
- Accelerate your DevOps - Think of IaC as a means to drive value to the end user faster through automation. Use IaC to iterate through versions of your product faster, be ready to scale, and build more reliable systems.
- IaC and GitOps - If you’re just starting off with IaC, build it with collaboration in mind. Treating your infra as code makes it possible to plan, run tests, and replicate your environments faster. Managing access effectively and setting expectations on new configurations can also help keep your costs under control.
With Argonaut’s modern deployment platform, you can get up and running on AWS or GCP in minutes, not weeks. Our intuitive UI and quick integrations with GitHub, GitLab, AWS, and GCP make managing your infra and applications easy - all in one place. Easily deploy third-party apps to go even further. All your infrastructure is automatically generated using terraform for full control and visibility.
Get started for free today!