AWS Config - powerful tool but a deployment headache
Seasoned users of AWS will be familiar with the AWS Config service. Config is a powerful tool that lets you continuously monitor and record the configuration of AWS resources in your account. Using Config, you can easily see changes to configuration of a given AWS resource on a timeline, see who made a given configuration change, and see relationships between resources in your account. Config also offers a powerful feature that allows you to setup governance rules to alert you when resources don't match your desired guidelines.
AWS recommends setting up AWS Config in each of your AWS account(s) as a best practice. Experienced users of Config know that this can rapidly become a cumbersome process if you have more than a few AWS accounts. AWS Config is a regional service which means you need to enable it in every region in which you operate. This can be a mind-numbing process using the AWS Console, and even using AWS CLI it is a complicated multi-step affair:
- Create an S3 bucket
- Create an IAM Role
- Create an SNS topic
- Create configuration recorder
- Create a delivery channel
- Start the Configuration recorder
Now imagine doing this doing this twenty times (once for each region) across 150 AWS accounts and I think you can begin to imagine the pain and suffering involved.
For admins of large multi-account environments that live within an AWS Organization, I'm happy to share there is a much easier way to deploy Config across all accounts and regions at once and for new accounts in your organization to have Config enabled automatically. The magic? CloudFormation StackSets to the rescue!
The Solution
In this solution, we will be setting up an AWS Organization wide Config service deployment, with all configuration recorder data being logged to a single centralized Amazon S3 bucket. Config service will be enabled in every account and region leveraging CloudFormation StackSets.
Prerequisites
In order to deploy this solution successfully there are a few requirements that must be taken care of first.
AWS Organizations
In order to leverage AWS StackSets, your AWS accounts must be members of an AWS Organization. Your accounts should be organized into one or more Organizational Units (OUs). I also recommend creating a "Testing" OU where you can try out your StackSet deployments on a test account before unleashing them on all accounts in your organization. In addition your AWS Organization should have all features enabled, not just consolidated billing.
Enable Stack Sets Trusted Access
We will be creating a StackSet with "service-managed" permissions. I highly recommend this option because with service-managed permissions, you do not need to worry about creating IAM roles in each account you will be deploying in...StackSets will take care of that for you automatically. This AWS documentation page provides more details on the different permissions models for StackSets deployments. Before you can start deploying StackSets with service-managed permissions, you must enable trusted access with AWS Organizations. There are multiple ways to enable this, but the simplest way to do so is via the CloudFormation console.
- Sign into your Organizations management account as a user with Administrator Access
- Navigate to the AWS CloudFormation console
- In the left-side navigation pane, choose StackSets. If trusted access is not currently enabled, a banner will appear at the top of the page that prompts you to enable trusted access.
- Click the Enable trusted access button to enable this feature.
Trusted access is successfully enabled when the following green banner displays
Centralized S3 Bucket
Rather than deploying an S3 bucket for each linked member account, this architecture relies on a single S3 bucket that all accounts write their logs to. If you are managing a large multi-account environment, you probably already have a centralized logging account, and this is where you should place this new bucket for config logging. Create an appropriately named S3 bucket in your logging account and ensure that the Block Public Access settings are enabled for the bucket. Then apply a bucket policy on the bucket that allows every account in the organization to access the bucket as well as the AWS Config service to access the bucket.
Below is a example bucket policy that demonstrates this configuration by leveraging the aws:PrincipalOrgID conditional to allow access from every account in the org and then uses Principal "Service": "config.amazonaws.com" to allow AWS Config the access it needs to the bucket.
{
"Version": "2008-10-17",
"Id": "Policy1357935677554",
"Statement": [
{
"Sid": "ListBucketAccess",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::my-config-bucket",
"arn:aws:s3:::my-config-bucket/*"
],
"Condition": {
"StringEquals": {
"aws:PrincipalOrgID": "o-1111111111"
}
}
},
{
"Sid": "PutObjectAccess",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my-config-bucket/AWSLogs/*",
"Condition": {
"StringEquals": {
"aws:PrincipalOrgID": "o-1111111111"
}
}
},
{
"Sid": "AWSConfigBucketPermissionsCheck",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::my-config-bucket"
},
{
"Sid": "AWSConfigBucketExistenceCheck",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::my-config-bucket"
},
{
"Sid": "AWSConfigBucketDelivery",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my-config-bucket/AWSLogs/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
With these prerequisites out of the way, we can now proceed with deployment of the solution using CloudFormation StackSets.
StackSet Deployment
StackSets are created in the AWS Organizations management account, using the same CloudFormation template concept as a normal CloudFormation stack. A StackSet is created and targeted at one or more Organizational Units (OUs), and CloudFormation deploys a Stack in each targeted account and region.
For this deployment, we will be leveraging a pre-built template from AWS, making our job easier. Follow this procedure to create a new CloudFormation StackSet in your AWS Organizations Management account
- Log into the AWS Console in your Management account with a user that has Administrator Access permissions
- Navigate to the CloudFormation service console
- In the left-side navigation pane, choose StackSets.
- Click the orange Create StackSet button to start the new StackSet deployment workflow
Below I will walk you through each step of the deployment workflow
Step 1: Choose a template
Ensure Service-managed permissions is selected and select the option to Use a sample template. From the Sample templates drop down, select the option "Enable AWS Config with central logging".
Then click the orange Next button.
Step 2: Specify Stack Set Details
Give your StackSet a meaningful name. You can leave all Parameters as default, with the following exception.
- Under Central S3 bucket, enter the Name of the S3 bucket you created to store your config logs. NOTE: do not enter an ARN here, just the name of the bucket
Optionally, you can modify the following:
- You can choose to include global resource types by changing that setting to true.
- You can modify the frequency of with which AWS config delivers configuration snapshots by modifying the Snapshot delivery frequency.
- You can specify an email address for AWS Config notifications in the Notification Email field.
When finished configuring options, click Next.
Step 3: Configure Stack Set Options
Apply any tags that you want for your StackSet in this screen. You can leave Managed Execution set to Inactive or change it to Active if you want CloudFormation to perform non-conflicting operations concurrently. When finished click the orange Next button.
Step 4: Set deployment options
Leave the deployment option set to Deploy new stacks.
For deployment targets, I always recommend starting out with a testing OU when deploying new StackSets to make sure there are no issues before targeting the entire organization. Select the radio button for Deploy to organizational units and enter the AWS OU ID in the text box. (You can retrieve OU IDs from the AWS Organizations console).
Leave Automatic deployment set to enabled, in this way if a new account is added to the OU a new stack is automatically pushed to that account. Account removal behavior should be set to delete stacks.
In the Specify Regions section, you can choose the regions you want to target from the drop down menu or click the Add all regions. Note that you should only target regions that are enabled in your target accounts, and those regions must also be enabled in your management account. See this note from AWS for more details on this.
Finally for Deployment options, choose the Maximum concurrent accounts you wish the stack to be deployed to at once. You can leave this at 1 or set a higher value.
For Fault tolerance, I recommend setting this a value higher than 0, otherwise your whole deployment will stop if a single account fails in a single region. I typically set this value to 2 or 3 so that a single error will not cause the entire StackSet deployment to fail.
Region concurrency controls how many regions will be deployed at the same time for a given account. For faster operations, set this to Parallel.
When finished configuring options, click Next.
Step 5: Review
Review all your settings to ensure they are accurate, then click the orange Submit button. You will be taken to the Operations tab for the new StackSet you just deployed, and it should have a status of Running
If all goes well, the Operation status will change from Running to Succeeded, indicating the stack was deployed successfully in all targeted accounts and regions.
You can then click on the Stack Instances tab to see the individual CloudFormation stack deployments in each account and region. You can also use the Stack Instances tab in the case of a deployment failure to see what went wrong.
Conclusion & Next Steps
To wrap up, in this post I've shown you how you can automatically deploy the AWS Config service across all regions for all AWS accounts in your AWS Organization, and have all configuration recorder data stored in a centralized S3 bucket. By deploying this architecture you will have the ability to assess, audit and evaluate the configuration of all AWS resources in your account(s), and track changes to those resources over time. In addition with this service enabled across your accounts, you can then implement additional governance across your AWS environments by leveraging AWS Config Rules.
Some of you may have already deployed AWS Config across your accounts in a decentralized fashion, and may be wondering how to migrate to this centralized architecture. Stay tuned as I will be covering that in my next post in this series.
I hope you've found this content useful. If you have feedback or suggestions on how this content could be improved, please leave a comment below.
Learn More
If you want to learn more about AWS Config service here are some links for more research:
- AWS Config Developer Guide
- AWS Config FAQs
- AWS Config Rules Development Kit @ Github
- AWS re:Invent 2018: Centrally Monitoring Resource Configuration & Compliance
- Monitoring and Remediating Non-Compliant Resources with AWS Config @ AWS YouTube
- Visualizing AWS Config data using Amazon Athena and Amazon QuickSight
Top comments (1)
Thanks for writing such an in-depth AWS post @cloudyadvice! I've marked this as a featured post on The Ops Community home page, so that it reaches more members.