Automation as key to cloud adoption success

After deploying several workloads in the public cloud, making mistakes, failing, fixing, and beginning using the cloud for production workloads, it is now the time to think about the next step in cloud adoption.

To be able to fully embrace the benefits of the public cloud, the scale, the elasticity, and the short time it takes to deploy new resources, it is time to put automation in place.

Automation allows us to do the same tasks over and over again, deploying the same configuration to multiple environments (Dev, Test, Prod) and get the same results – no human errors (assuming you have tested your code…)

Automation can be achieved in various ways – from using the CLI, using the cloud vendor's SDK (languages such as Python, Go, Java, and more), or using Infrastructure as Code (such as Terraform, AWS CloudFormation, Azure Resource Manager, and more).

In this article, we shall review some of the common alternatives for using automation using code.

Why use code?

The clear benefit of using code for automation is the ability to have change management. Simply choose your favorite source control (such as GitHub, AWS CodeCommit, Azure Repos, and more), upload your scripts and have the version history of your code, and be able to know at each stage who made changes to the code.

Another benefit of using code for automation is the fact that the Internet is full of samples you can find to automate (almost) anything in your cloud environment.

The downside of doing everything using code, is the learning curve required by your organization's IT or DevOps teams, learning new languages, but once they pass this stage, you can have all the benefits of the scripting languages.

Automation – the AWS way

If AWS is your sole cloud provider, you should learn and start using the following built-in services or capabilities offered by AWS:

Infrastructure as Code

• AWS CloudFormation – The built-in IaC for deploying and managing AWS resources.

  Reference: https://github.com/aws-cloudformation/aws-cloudformation-samples

• AWS Cloud Development Kit (AWS CDK) – Ability to write CloudFormation templates, based on common programming languages such as Python, Java, DotNet, and more.

  Reference: https://github.com/aws-samples/aws-cdk-examples

Policy as Code

• Service control policies (SCPs) – Managing permissions in AWS Organizations.

  Reference: https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps_examples.html

CI/CD pipeline

• AWS CodePipeline – A fully managed continuous delivery service.

  Reference: https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials.html

Containers and Kubernetes

• Amazon ECS – Container management service based on the AWS platform.

  Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/example_task_definitions.html

• Amazon Elastic Kubernetes Service (EKS) – Managed Kubernetes service.

  Reference: https://github.com/aws-quickstart/quickstart-amazon-eks

Automation – the Azure way

If Azure is your sole cloud provider, you should learn and start using the following built-in services or capabilities offered by Azure:

Infrastructure as Code

• Azure Resource Manager templates (ARM templates) - The built-in IaC for deploying and managing Azure resources.

  Reference: https://github.com/Azure/azure-quickstart-templates

• Bicep – Declarative language for deploying Azure resources.

  Reference: https://github.com/Azure/azure-docs-bicep-samples

Policy as Code

• Azure Policy – Enforce organizational standards across the Azure organization.

  Reference: https://github.com/Azure/azure-policy

CI/CD pipeline

• Azure Pipelines – A fully managed continuous delivery service.

  Reference: https://github.com/microsoft/azure-pipelines-yaml

Containers and Kubernetes

• Azure Container Instances - Container management service based on the Azure platform.

  Reference: https://docs.microsoft.com/en-us/samples/browse/?products=azure&terms=container%2Binstance

• Azure Kubernetes Service (AKS) – Managed Kubernetes service.

  Reference: https://github.com/Azure/AKS

Automation – the Google Cloud way

If GCP is your sole cloud provider, you should learn and start using the following built-in services or capabilities offered by GCP:

Infrastructure as Code

• Google Cloud Deployment Manager - The built-in IaC for deploying and managing GCP resources.

  Reference: https://github.com/GoogleCloudPlatform/deploymentmanager-samples

Policy as Code

• Google Organization Policy Service - Programmatic control over the organization's cloud resources.

  Reference: https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints#how-to_guides

CI/CD pipeline

• Google Cloud Build - A fully managed continuous delivery service.

  Reference: https://github.com/GoogleCloudPlatform/cloud-build-samples

Containers and Kubernetes

• Google Kubernetes Engine (GKE) – Managed Kubernetes service. Reference: https://github.com/GoogleCloudPlatform/kubernetes-engine-samples

Automation – the cloud agnostic way

If you plan for the future, plan for multi-cloud. Look for solutions that are capable of connecting to multiple cloud environments, to decrease the learning curve of your DevOps team learning the various scripting languages and being able to deploy workloads on several cloud environments.

Infrastructure as Code

• Hashicorp Terraform – The most widely used IaC for deploying and managing resources on both cloud and on-premise.

  Reference: https://registry.terraform.io/browse/providers

Policy as Code

• Hashicorp Sentinel – Policy as code framework that compliments Terraform code.

  Reference: https://www.terraform.io/cloud-docs/sentinel/examples

CI/CD pipeline

• Jenkins – The most widely used open-source CI/CD tool.

  Reference: https://www.jenkins.io/doc/pipeline/examples/

Containers and Kubernetes

• Docker – The most widely used container run-time for deploying applications.

  Reference: https://github.com/dockersamples

• Kubernetes – The most widely used container orchestration open-source platform.

  Reference: https://github.com/kubernetes/examples

Summary

In this post, I have reviewed the most common solutions that allow you to automate your workloads' deployment, management, and maintenance using various scripting languages.

Some of the solutions are bound to a specific cloud provider, while others are considered cloud agnostic.

Use automation to fully embrace the power and benefits of the public cloud.

If you don't have experience writing code, take the time to learn. The more you practice, the more experience you will gain.

As Werner Vogels, the Amazon CTO always says – "Go Build".

About the Author

Eyal Estrin is a cloud and information security architect, the owner of the blog Security & Cloud 24/7 and the author of the book Cloud Security Handbook, with more than 20 years in the IT industry.

You can connect with him on Twitter and LinkedIn