The Ops Community ⚙️: The Ops Community

Has Generative AI ruined the re:Invent experience?

Mike Graff — Thu, 03 Oct 2024 21:09:58 +0000

Introduction

I've been attending re:Invent every year since 2015, and while the conference has its share of issues, I've always returned because I've felt the benefits far outweigh the drawbacks. It seems as if Generative AI related sessions have taken over re:Invent, to the detriment of other topics.

AWS re:Invent - a learning conference

AWS re:Invent is...a lot to deal with. Yes, the session catalog is terrible to navigate. So much so that it has inspired some fellow Community Builders to step in with the creation of third party tools to fill the gap. Sure, the session registration system is a total mess and has been for years. Not to mention it completely disadvantages folks from different countries that live in different time zones. Lastly, the huge crowds and navigating across multiple venues up and down the strip makes things logistically painful.

However at the end of the day it has always been the high quality and deeply technical nature of the content, the variety of topics covered, and the outstanding networking opportunities that kept me coming back year after year. (Not to mention the fantastic musical performances at re:Play).

However in 2024 Generative AI has become the dominant topic in the tech world. It has sucked all the oxygen out of the room at most tech conferences, and that trend seems to have extended to re:Invent as well. Let's dive in...

Generative AI related sessions have taken over re:Invent

I'm always excited when the re:Invent session catalog gets released, and this year was no different. However that excitement diminished significantly when I started looking at the breakdown of session subjects. I quickly discovered that the catalog is dominated by AI topics. More worrisome, several of the areas I'm passionate about (like Networking and Storage) have a shockingly low number of sessions this year. Here's a breakdown of the 2378 sessions currently listed in the re:Invent catalog by session topic:

Topic	Number of Sessions	Percentage of Total
AI/ML	820	34%
Analytics	223	9%
Architecture	243	10%
Business Applications	129	5%
Cloud Operations	237	10%
Compute	153	6%
Content Delivery	31	1%
Customer Enablement	34	1%
Databases	138	6%
DevOps & Developer Experience	214	9%
End User Computing	18	0.8%
Hybrid Cloud	37	2%
IoT	65	3%
Kubernetes	77	3%
Migration & Modernization	234	10%
Networking	70	3%
New to AWS	46	2%
Security, Compliance, and Identity	227	10%
Serverless & Containers	185	8%
Storage	111	5%
Training & Certification	29	1%

re:Invent 2024 session catalog breakdown by topic area. Note that sessions are often tagged to more than one topic area.

As you can see AI/ML topics currently make up whopping 34% of the session catalog. In addition, because AWS will tag the same session with multiple topics, a fair number of the sessions in other topic areas are also AI related.

Historical Trend

The AI takeover trend is even better illustrated when you compare the current session catalog to the catalogs from the past few years. (A big shout out to my fellow AWS Community Builder Raphael Manke for providing 2022 and 2023 session catalog data). Take a look at the trend graph for the various topic areas:

Here's a further detailed breakdown of selected session topic areas and the percentage change from 2022 to 2024:

Topic Area	2022	2023	2024	Change 2022 to 2024	Percentage change
AI/ML	417	768	820	403	52%
Architecture	314	309	243	-71	-23%
Cloud Operations	367	304	236	-131	-43%
Compute	237	209	153	-84	-40%
Customer Enablement	165	76	34	-131	-172%
Databases	272	200	137	-135	-68%
DevOps	370	263	214	-156	-59%
EUC	67	33	18	-49	-148%
Hybrid Cloud	164	82	36	-128	-156%
Networking & Content Delivery	238	128	101	-137	-107%
Serverless & Containers	311	292	185	-126	-43%
Storage	291	168	112	-179	-107%
Training & Certification	102	92	30	-72	-78%

re:Invent Session Catalog for the past three years, broken down by topic area

The number of AI/ML sessions has essentially doubled since 2022. Now I can't really blame AWS for this, as I already stated that is where all the buzz has been for the past 18 months so it makes sense. However, my real concern is the precipitous drop in some key topic areas that are essential to building and working in the cloud computing.

My typical go to topic areas at re:Invent have been Architecture (23% decrease), Cloud Operations (43% decrease), Networking (107% decrease) and Storage (107% decrease). The huge decreases in the areas of Customer Enablement and Hybrid Cloud are also alarming for folks who are coming to re:Invent for the first time and trying to figure out their cloud migration plans.

Conclusions

Look, I get it, Generative AI is the new hotness, and AWS has to react to that given how often they are getting beat up for being "behind in AI." However there needs to be balance, as a large number of folks who are building on AWS (including me) are not building anything that has to do with AI. If we are going to spend $2000 and a week of our lives suffering through Las Vegas, our time needs to be rewarded with a plethora of deep technical sessions on the topics that we care about.

Cloud Economist (and master of cloud snark) Corey Quinn had a blog post reviewing the AWS New York summit earlier this year and I found this quote summed things up nicely:

And so, the hyperfocus on GenAI is concerning to me because of what’s being shunted aside to create room for it. They’re Amazon WEB Services, not Amazon GenAI Services. I fix large AWS bills for large enterprises for a living; my customers have a raft of very large-scale challenges that don’t involve GenAI in the slightest.
- Corey Quinn

Will I be at re:Invent this year? Absolutely, for the reasons I outlined at the start of this post. Will this be my last re:Invent? I'll let you know after December.

What are your thoughts on the makeup of the re:Invent session catalog? Am I overreacting? Let me know in the comments.

Use IAM Roles Anywhere to reduce the use of IAM keys

Mike Graff — Mon, 06 Nov 2023 00:30:38 +0000

Abstract

Exposed static IAM keys are one of the most common security risks for AWS accounts. Avoiding the use of static keys is a best practice that can be hard to achieve in hybrid cloud environments where access needs to be given to external systems. IAM Roles Anywhere is a service that help mitigate this risk. In this article I detail the problems and risk associated with static keys, how IAM Roles Anywhere can solve this problem, and walk through how to setup the complete solution.

The Keys Problem
- Why this is bad
- The Toil of Managing Keys
- What about IAM Instance Roles?
- No Solution for External Systems
IAM Roles Anywhere Solves the Keys Problem
How does IAM Roles Anywhere Work?
- IAM Roles Anywhere Components
- IAM Roles Anywhere Architecture
Setting Up IAM Roles Anywhere
- Setup AWS Certificate Manager Private CA
  - Steps to Setup the Root CA
  - Important Note on CA Hierarchies
- Configure IAM Roles Anywhere
  - Establish Trust Anchor
  - Configure Role
  - Configure a Profile
Configuring the External System
- Generate Client Certificate
- Export the Certificate
- Decrypt the Private Key
- Install IAM Credentials Helper
Making the Connection
- Manually request and set credentials
- Leverage Roles Anywhere in AWS CLI Config
Closing Thoughts
References

The Keys Problem

One of the more pesky problems to solve when managing an AWS environment is granting access to external systems in a secure manner. External access to the AWS console can be handled via username and password (or even better, leverage SSO login via IAM Identity Center!). Unfortunately, this method doesn't work for API access from machines and applications outside your AWS environment.

Why this is bad

The most common solution for providing external access is not awesome. You generate an IAM access key pair for the application or system and pass those credentials over to your developer for use. Afterwords, you keep your fingers crossed that they do not share these keys with other folks or store them in plain text on their computer. Even worse would be if they put these credentials directly into their code and then commit it to a git repo.

Without a doubt this last scenario happens all the time, and consequently it is something that hackers are constantly scanning GitHub repos to find and exploit to get access to your AWS accounts.

The Toil of Managing Keys

Let's say you are able to avoid these credentials getting exposed accidentally. You still have to make sure you are performing good security practices on these keys by rotating them regularly. For example, in my environment we rotate keys every 75 days. Then you have to go through the process of securely exchanging the keys with your developers again, and they have to update their application to use the new keys. Wash, rinse, and repeat this process every few months and that adds up to a lot of toil just to keep your keys secure.

What about IAM Instance Roles?

This problem has been solved for years for systems running inside of AWS with IAM Instance Roles. This feature allows you to assign an IAM Role directly to an EC2 instance (or container or Lambda function). As a result, the EC2 instance then dynamically retrieves an access token for the role via the EC2 Instance Metadata service. Very slick and no static keys required!

No Solution for External Systems

However, the very nature of the way this service works requires that the calling system be inside AWS. For external systems you had no option beyond static keys. Thankfully, that all changed in July 2022 when AWS announced IAM Roles Anywhere.

IAM Roles Anywhere Solves the Keys Problem

IAM Roles Anywhere makes it possible to use IAM Roles on systems outside of AWS. It provides a mechanism for external servers, containers, and applications to obtain temporary AWS credentials in a manner similar to EC2 Instance Roles. Thus it eliminates the need to create and manage static access keys and dramatically improves the security posture of your AWS accounts.

How does IAM Roles Anywhere Work?

IAM Roles Anywhere leverages public key infrastructure (PKI) as a mechanism to establish trust between your external system and your AWS Account. Systems sitting outside of AWS hold X.509 Certificates that they present as part of a CreateSession request. Next these certificates are validated by IAM Roles Anywhere and then a temporary set of credentials are returned to the client.

IAM Roles Anywhere Components

There are six basic components to the IAM Roles Anywhere architecture.

Certificate Authority (CA). The CA is the heart of your public key infrastructure and is responsible for issuing certificates. For IAM Roles Anywhere you can use a CA provided by Amazon Certificate Manager, or you can use an existing External CA.
Certificates are digital documents that securely associate cryptographic key pairs with a system. A certificate contains the public key and a signature that has been encrypted with the private key. This allows a third party to verify that a certificate is valid by decrypting the signature using the public key. Certificates also contain a trust chain that links the certificate back to the CA that issued it.
A Trust Anchor is used to establish a trust relationship between IAM Roles Anywhere and your Certificate Authority.
An IAM Role. You probably already know that a Role is an IAM identity that contains specific permissions you wish to grant and that can be assumed by anyone who needs it. In order to use a role with IAM Roles Anywhere, your role must be configured to trust the IAM Roles Anywhere service principal.
Profiles are used to specify which roles IAM Roles Anywhere can assume and what your workloads can do with the temporary credentials that are issued. You can specify a session policy to limit the permissions created for the session.
Lastly, the IAM Roles Credential Helper is a downloadable tool provided by AWS that allows you to request temporary security credentials via the CreateSession API.

IAM Roles Anywhere Architecture

Here is a diagram that illustrates the components of the IAM Roles Architecture:

The components of the architecture work together in the following way:

A Trust Anchor is established between IAM Roles Anywhere and the Certificate Authority (CA). This CA can be running inside of AWS or externally in your own data center or even another cloud provider.
The CA issues an x.509 Certificate to the external system, where it is installed in the local store.
The IAM Role is created and configured to trust the IAM Roles Anywhere service principal.
A Roles Anywhere Profile associates the IAM Role with Roles Anywhere and can set session restrictions if desired.
The External Server issues a CreateSession request and provides it's Certificate along with specifying the role it wishes to assume.
IAM Roles Anywhere validates the Certificate is valid and tied to the CA contained in the trust anchor.
Once these validations are complete, the system is now authenticated and IAM Roles Anywhere will create a role session via STS and pass the session credentials back to the external system.
The external system can now use these temporary credentials to make any AWS API calls that are allowed in the IAM Role.

Setting Up IAM Roles Anywhere

Now that I've gone over the basic architecture of IAM Roles Anywhere, next I'm going to walk you through an end to end deployment of this solution. For my lab deployment I will use AWS Certificate Manager Private CA as our Certificate Authority. As a result the first thing I will walk through is how to stand up the Private CA.

Setup AWS Certificate Manager Private CA

AWS Certificate Manager is AWS' fully managed Certificate Authority service. You are probably most familiar with this service as a method for issuing public certificates for use with other services like CloudFront and API Gateway, but the service also has the capability to act as a Private CA, allowing you to setup entire managed CA hierarchies without the headache of managing your own CA (and trust me, having setup my own private PKI hierarchies in the past, it neither fun to setup nor maintain).

Steps to Setup the Root CA

First, we need to setup the Root CA

To get started, login to your AWS account with the appropriate credentials and navigate to the Certificate Manager service. From the Certificate Manager home page, click the link for AWS Private CA on the left hand navigation bar

On the resulting page, click the orange Create a Private CA button to get started creating your new Private CA.

After clicking the button, you will be presented with the Create a private certificate authority wizard page. Here we will be configuring all the required options to stand up our private Certificate Authority.

For your CA Mode options, select General Purpose. NOTE: if you are following along and setting this up in a lab environment be aware of Amazon Private CA pricing. A General Purpose private CA will cost you $400/month! I have not tested the option to use the short-lived certificate mode, and it certainly would not work for a production deployment as the certificates would expire too quickly.
For CA type, select Root
Fill in your Subject DN options as you please with Organization, Organization Unit, Country, State, Locality and Common Name.
Set your key algorithm as desired, e.g. RSA 2048

Next we need to setup a Certificate Revocation List distribution point so we can have a place to publish certificates that are no longer valid. All properly configured CAs must have a CRL location for the CA to be trusted.
Check the boxes for Activate CRL Distribution and Create a new S3 bucket.
Enter a new bucket name in the S3 bucket name field.
You can leave the other CRL settings as default.
Add tags as desired
Under CA permissions options, ensure that the check box for Authorize ACM access to renew certificates is checked.
Finally, check the box acknowledging the pricing for ACM and click the Create CA button.

Our Root CA is now created, but to finish activating it we need to Install a CA Certificate. From the Actions menu in the top right corner of the Certificate Authority homepage, you should choose Install CA Certificate. On the resulting screen accept the default 10 year validity period and SHA256 signing algorithm and click the Confirm and install button.

After completing these steps, you now have a functioning Root CA that is able to start issuing private certificates.

Important Note on CA Hierarchies

From a PKI best practices perspective, you normally would never issue client certificates from your Root CA. Instead you would setup a subordinate "Issuing CA" that trusts the Root CA and that would be used to issue all your client certificates. However in the interest of simplifying this blog post and avoiding the additional cost of running multiple CAs, I'm going to skip that step here and move on to configuring IAM Roles Anywhere. Just be aware of this for your production Certificate Authority deployment.

Configure IAM Roles Anywhere

Now that we have a functioning Certificate Authority, let's get into setting up IAM Roles Anywhere. To get started with this task, navigate to the IAM service page in your AWS Console. Next, click on Roles on the left side navigation bar. At the bottom of the resulting page you will see a section for Roles Anywhere. Click the Manage button to get started setting it up.

Establish Trust Anchor

The Roles Anywhere page has a very nice Setup wizard that will walk you through the process. The first thing we need to do is create a trust anchor to the Private CA we setup in the previous step. Click the orange Create a trust anchor button to get started.

You will be taken to the Create a trust anchor wizard page.

First, enter a friendly name for your trust anchor.
Next you will choose your CA source. Here is where you can specify an external CA to trust via uploading a certificate bundle, or you can go with AWS Private Certificate Authority, which is the default option.
Below in the next section you will see a list of AWS Private Certificate Authorities in the account. Click the radio button next to the root CA we created in the previous step.
Leave notification options as default and add tags as appropriate.
Finally, click the orange Create a trust anchor button to complete the process.

Configure Role

After you create your trust anchor, the Setup wizard will move to the next step, Configuring roles. If you have an existing role, you can easily configure it for Roles Anywhere by adding the required trust statements to the role. For our demo, let's make things simple and create a new role for use with Roles Anywhere. You can do this by clicking the Create a new role button.

Clicking that button will take you directly to the IAM Role creation wizard. Step 1 is Select trusted entity.

Click the button next to AWS Service.
From the Use case drop down menu, select Roles Anywhere.
Click Next.

From here we need to add permissions to the role. This will control what API actions the external system is able to use via Roles Anywhere.

You can attach any existing IAM policy in your account, AWS Managed policies, and even attach multiple policies.
For this demonstration I'm going to grant the AmazonS3ReadOnlyAccess managed policy and click Next.

To complete the wizard, give your new Role a name and description, review the trust policy and permissions you just setup, assign tags and then click the orange Create Role button.

Configure a Profile

After you role is created, navigate back to the Manage Roles Anywhere screen. The final setup step is to create a Profile that will associated your IAM Role for use with Roles Anywhere using a role session policy. To complete this step, click the button that says Configure a Profile:

Time for yet another wizard page! Here you will complete the following steps:

Give your Profile a name
In the Roles section, chose the role we created in the previous step
In the Session Policies section, you can optionally limit what permissions are granted by the role for the session created via Roles Anywhere. For our case we will remove the default inline policy that is present, as it grants ALL access.
Apply Tags
Finally click the Create a profile button to finish the wizard.

At this point we have successfully completed all the setup tasks to get our IAM Roles Anywhere backend configured.

Configuring the External System

With our Roles Anywhere backend up and running, let's move on to configuring the external system for access. The first step for this is to issue an x.509 certificate for our external system to use for authenticating to IAM Roles Anywhere.

Generate Client Certificate

In the AWS console, navigate back to Amazon Certificate Manager service. From the service home page, click Request Certificate on the left side navigation bar (or click the big red orange button on the right):

Under Certificate type, choose the option Request a private certificate and click Next:

You guessed it, it's wizard time! From this wizard we will be completing the required properties for our certificate request (CSR).

In the Certificate Authority drop down menu, select the private CA we created earlier.
Enter your domain name for the certificate in the Domain Name section.
Choose your Key algorithm. In this case I'm going with the default of RSA 2048.
Add tags as desired.
Under Certificate Renewal Permissions, click the box to acknowledge that ACM will need permissions to rewew this cert. (you may remember we already granted this permission when we setup the CA originally).
Click the orange Request button to complete the request.

Export the Certificate

Your certificate will get created and after a few moments the certificate status should update from Pending validation to Issued in the Certificates list. Once the certificate is issued, we need to download it so we can place it on our external system. Click on the certificate in the list and from the details page click the Export button.

On the Export Certificate page you will need to add a passphrase so that the private key can be encrypted. Next, click the checkbox to acknowledge that you will be charged for exporting this certificate:

After you click Generate PEM encoding, the service will return to you three items, each with Download links:

Certificate body
Certificate chain
Certificate private key

Now you will need to download each of these to your local system. You can do this by either clicking the download buttons for each item, or creating text files on your local system and copying the text into them. Place all these files in a folder on your system. After downloading the files, you will need to rename certificate.txt to certificate.pem and private_key.txt to private_key.pem.

Decrypt the Private Key

Once the files are downloaded, next we need to decrypt the private key using the passphrase we entered when we performed the export. On a Mac or Linux system we can do that easily with openssl. Here is the command to use:

openssl rsa -in private_key.pem -out decrypted_private_key.pem

Openssl will run and prompt you to enter the passphrase you created, and then decrypt the key to the designated output file:

Enter pass phrase for private_key.pem:
writing RSA key

Success! We've now created a new certificate for our system in Amazon Private CA, exported it our system have it ready for use with the IAM Roles Credential Helper.

Install IAM Credentials Helper

As mentioned before the IAM Roles Credentials Helper is a downloadable tool maintained by AWS. The tool is used to create a SigV4 signature with your certificate and make a call to the Roles Anywhere endpoint to obtain session credentials. Next, the Roles Anywhere endpoint then returns the credentials to the calling process in JSON format. You can download the latest version of the tool from the IAM Roles Anywhere User Guide, where there are packages for Linux, Windows, and Darwin (MacOS). Download the helper and place it in the same directory where you placed your certificate and private key. First, you'll need to make the package executable before you can use it:

chmod +x aws_signing_helper

With this last step done, we've completed all the end to end setup work needed to use Roles Anywhere. To wrap up the article, I'm going to demonstrate how to make a CreateSession API request using the credentials helper.

Making the Connection

There are three things we will need to have in order to use the AWS IAM Roles Anywhere Credentials Helper to make a CreateSession request.

Roles Anywhere Trust Anchor Amazon Resource Name (ARN)
Profile ARN
IAM Role ARN

The first two items can be obtained from the IAM Roles Anywhere management page, while the IAM Role ARN can be found by examining the details of the Role in the IAM Roles page. You will also need the filenames for your certificate and decrypted private key that we downloaded in the previous step.

Manually request and set credentials

You will combine all these items as the options for our rather long aws_signing_helper command:

./aws_signing_helper credential-process \
--trust-anchor-arn arn:aws:rolesanywhere:us-east-1:111111111111:trust-anchor/49d455a6-deec-4cfc-9c12-2c75217ea49a --profile-arn arn:aws:rolesanywhere:us-east-1:111111111111:profile/d0119c28-ecfa-4ad2-88c7-cacfd61bb268 \
--role-arn arn:aws:iam::111111111111:role/cloudyadvice-roles-anywhere-role \
--certificate certificate.pem --private-key decrypted_private_key.pem

Note I've modified the account numbers here, you should replace with the proper values from your ARNs. If all goes well, AWS will return you an Access Key ID, Secret Access Key, and Session token in JSON format, like so:

{"Version":1,"AccessKeyId":"ASIAVFPLXXXXXXXXXXXX","SecretAccessKey":"+ZCVCbiRHXwCOBZCi0JVv/XXXXXXXXXXXXX","SessionToken":"IQoJb3JpZ2luX2VjEC4aCXVzLWVhc3QtMSJGMEQCIFgE1/1OZjanZDOiMnVkxgKMXsoZVlQl3QJy2POXadb/AiAm7EqPHvXslugCqcaS6wCtquf/XIaMTg1YbJg3ywfXDCqIBAhmEAIaDDM1NTM2NDgyOTg2NCIM0BYAV8uBV1afYXAuKuUDMXS8VhT4SnBqg+S4yqtOj0TSZdbp/z4AKJeovB6u9g/5f2SA1tAG3hkByq/PEiuLFGN0kIgkWT66CPDNHS5iuPVXhiHIghaNqVDXzGXvWcxp5SPLbIIAQxfyQTmNiBhqg82VDaEUEutSh2kd8CdvTtlP39tEFE0q8poWbNCnDCwg1Wj50LaH7IQb3jYoTxHAnHEkTmtlqmoeYIM7UVc1RQ2Fox37nnMZVqaeY0tCyxyfmQn8tCEmaFRTxYWweCfxB7WIjptGwQGUchEBbISHnJulKU9Pv8lgct3HENWqyQcVKSWRZDvIAuFjpfq8cahcAVHrzM20KNMO3OxfZyXhBgSNalxO6c0pGoBDgmlH1o8yEVky2GSzYncSwyL++KPAzb/+FeNiT4aJeSyd3ns0Ewp/s30JbC8fuID6UuuuSfNa7VetSFBanSfIkjDhUnefjyYFyz2f+7k2X5HPYeAmT/zjwOGpNY7KcIJqZQHERhaMWn5Gy0RvjvAoVVAgyEwqwBff76TEwmxh6DMN4o/LRuOnflommJojT0BmVtfBD5Ykp53xZXPCQctF7ijN40Ud18w8ZIBF/6QUAdQe1Lp32kIH9HPq3Lg4YlvJzdsMX6BXemDWHCFLCE1BfFQXXBRRs0p8Ci8wgueaqgY6lAHD/crexcd3uBsB87czp6kyYV8iH4BMPPpxbIs3fAgjOkJMta2pROF72YKR9FRF9ixciFkBCpnACjWGSDSuwx/+elwoDJ3zlCOad7SYX0J8CBxAKKAQ564MOiYrB9wdf3N2sQd+dKJkmPSZxEN1sWnMnD/D+HrWB7A0ULsW3phER50gisns8G/KsE4q4gtz1QOrOIP6","Expiration":"2023-11-04T22:11:30Z"}%

As you can see, Roles Anywhere has sent you back a send of temporary credentials in the form of an Access Key ID, a Secret Access Key, and a Session Token. You can simply use these values returned here to set as temporary environment variables for your AWS credentials, and then issue calls to AWS using those environment variables. For example:

export AWS_ACCESS_KEY_ID=access-key-id-value
export AWS_SECRET_ACCESS_KEY=secret-access-key-value
export AWS_SESSION_TOKEN=session-token-value

Once that is done you can issue AWS CLI commands, e.g. get a list of S3 buckets:

aws s3api list-buckets

Which should return you a list of S3 buckets in JSON format.

Leverage Roles Anywhere in AWS CLI Config

Manually requesting credentials is a great way to test, but rather laborious to do on a regular basis. Fortunately you can leverage the aws_signing_helper as a custom credential_process for the AWS CLI. To do this, you simply need to add it as a new profile entry in your AWS CLI credentials file to invoke it. First, copy the certificate files and the aws_signing_helper executable to the same directory where your AWS CLI is located, typically ~./aws. Next, you add a new profile to your ~/.aws/config file that leverages IAM Roles Anywhere. The profile entry should look something like this:

[profile rolesanywhere]
credential_process = /users/cloudyadvice/.aws/aws_signing_helper credential-process --trust-anchor-arn arn:aws:rolesanywhere:us-east-1:111111111111:trust-anchor/49d455a6-deec-4cfc-9c12-2c75217ea49a --profile-arn arn:aws:rolesanywhere:us-east-1:111111111111:profile/d0119c28-ecfa-4ad2-88c7-cacfd61bb268 \
--role-arn arn:aws:iam::111111111111:role/cloudyadvice-roles-anywhere-role \
--certificate certificate.pem --private-key decrypted_private_key.pem

Obviously, you would need to replace the dummy values shown here with the proper values from your own environment. Once you've added this new profile to your ~/.aws/config file, you can call it for use with any AWS CLI command with the --profile flag, for example:

aws s3 ls --profile rolesanywhere

Now AWS CLI will leverage Roles Anywhere to mint dynamic temporary credentials when you execute commands using this profile switch. Totally awesome!!

Closing Thoughts

I was so excited when IAM Roles Anywhere was announced because it solves a real world problem I often have to deal with in my day job. As a result I was glad to finally got around to testing it out and writing up this blog post.

Obviously, this was a fairly long blog post covering a pretty complicated setup. AWS IAM by itself is a very complicated service. Then when you add public key infrastructure into the mix, things get even more complicated. To help with this complexity, I've done my best here to walk through the steps in an easy to follow fashion so you can start experimenting with this powerful service.

I hope that you will see the benefit of this great service and use it to make your own AWS environments more secure. Are you already using IAM Roles Anywhere in your environment? Share your experiences and thoughts in the comments!

References

AWS Security Blog - IAM Roles Anywhere Launch Post

AWS Security Blog - IAM Roles Anywhere with an external certificate authority

AWS Security Blog - Enable external pipeline deployments to AWS Cloud by using IAM Roles Anywhere

AWS IAM Roles Anywhere User Guide

YouTube - AWS re:Inforce 2023 Managing hybrid workloads with IAM Roles Anywhere, featuring Hertz

Hacker Loi YouTube - AWS IAM Roles Anywhere Full Tutorial

GitHub AWS repo - roles anywhere credential helper

Announcing some changes and saying goodbye ✌

Ella (she/her/elle) — Wed, 01 Mar 2023 19:39:37 +0000

First, an announcement:

Friends, as February draws to a close it's time to announce my departure from the Ops Community moderator team and I want to take this opportunity to thank you all.

Ever since I officially joined the team last summer, I've enjoyed getting to know our regular contributors through your content and comments - and in the process I've learned so much more about DevOps than I ever thought I could!

Thanks for welcoming me into the space, showing up to share your experiences, and answering my questions (or bearing with my DevOps ignorance!). I've had so much fun watching this community evolve as you help each other, gain new skills of your own, and build your own followings. Every moment has been a privilege.

We'd love your help!

That's right! You can help this community enter a new chapter:

Frequent readers will have observed that the Weekly Achievements series posts have grown longer as more posts are being published each week (which has given me so much more material to explore and digest, so thanks again!), and those of you who have been around for some time will have noticed the badges popping up in your profiles for reaching follower and publishing milestones. The time and attention that this growing community deserves is expanding with its readership.

This is the ideal time for you to take up the Tag Moderator or Trusted User role you've been thinking about!

We want to continue to celebrate excellent content, and to do that we need your expert opinions on the accuracy and validity of the content that is being shared. If you're a specialist in Azure, Pulumi, tutorials, observability, etc., or just want to help keep this community healthy, we would love to hear from you!

Drop the team a note at socials@ops.io or leave a comment below.

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 27 Feb 2023 20:43:10 +0000

Congrats on the launch of the new Blink website, @patrick_londa and @johnson_brad - it's such a fresh new look! What's next for y'all?

I'm wrapping up a few things ahead of a trip out to East Asia next week, so let's keep this week's round-up short and sweet 🍯

A warm welcome 👋

... to @rafaelonline and all our new members this week!

👉 If you're new here and would like some suggestions for people to follow who share your interests, we welcome you to share a bit about what you're working on and what you're like to learn and share! Drop your intro in our Welcome post here

ICYMI

Bonus points if you can pick up on a solid theme from the past week 🕵

@danielepolencic was all about the Kubernetes containers ✨
@eriklz separated out the Kubernetes (IaC) from the containers (Docker) ⚡
and @eyalestrin asked if we're ready for IPv6 then gave an answer involving Kubernetes and containers (sorry, no spoilers here - you'll have to read it for yourself!) 🔥

I swear, you can't make this up!

What are you planning for the week ahead?

👉 Let me and the community know by commenting below.

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 20 Feb 2023 20:17:33 +0000

If you've ever had any doubts about sharing your knowledge with this community or any other, @eyalestrin has something to say to you!

As a community mod, this post on why you should share your knowledge is one of my favorite posts of the year so far, because of all the reasons Eyal lists: I believe it to be true, that sharing your knowledge is an act of giving and receiving without ever having to take anything from anyone else.

Join our incredible community members listed below and try it for yourself - you may even end up in next week's community check-in post!

A warm welcome 👋 to all our new members!

@bernardo joined us this week and immediately made the Ops Community feel like their home-away-from-home with a trio of posts! Welcome, Luiz, and thank you for making your presence felt with your contributions - if you haven't already met @eyalestrin I think you're going to get along!

ICYMI

Luiz wasn't the only person delivering three posts this week. In fact, we had three people with three posts (hold on while I go buy a lottery ticket 🍀):

@oleonardorodrigues was on a posting roll this week with not one, not two, but three posts! 💪
@argonaut also delivered a trinity of posts with two primers: one on GitOps and the other on secret management, with a more detailed guide to managing Kubernetes Secrets with Hashicorp Vault 🤫

Meanwhile, away from the three-of-threes, it was business as usual:

@techielass kept the #azure content rolling with this summary explainer on Azure savings plans for compute, 💸
@danielepolencic guided us through shutting down pods in Kubernetes - I love the use of "gracefully" in tech, don't you? 💃
@devinterrupted talked with Steve Pereira about value stream management in DevOps

If you're planning to leave the house in 2023, then you'd better start signing up for some of these free events:

This week, @alvaradodaniel3 dropped New Relic's blog summary for January and there are some gems to check out in there, but if you want to hear from the New Relic team in person then opportunities are coming soon! Tomorrow, Tuesday 21 February at 5:30 PM ET, the team will be running a workshop in at the New Relic HQ in ATL. If you'd like to learn how to create an app with React Native and meet some devs in your neighborhood, you can sign up here. If Atlanta's a little too far north for you, O11y Day: Miami is taking place on Thursday 23 February - and if that's still too far north then maybe FutureStack São Paulo is closer to home on Wednesday 8 March. Is West the best for you? Then join FutureStack San Francisco on Wednesday 15 March.
For those of y'all who are nowhere near any of those stunning locations I just listed, the PagerDuty Community Team have got you covered with their virtual meetup: the Quarterly Terraform Roundtable. Join @lnxchk and the team to talk all things #PagerDuty and #Terraform! Date and time is Tuesday, February 21 at 10am PST, 1pm EST, 6pm UTC. Sign ups and all the info can be found here✨

What are you planning for the week ahead?

Did anyone else feel like last week somehow crept by in a weird way? I've spoken to a couple of people who had a similar experience as me: I felt like I was working at a good pace on things and making progress on my to-do list, but all of a sudden it was Friday and I wasn't finished!

👉 Let me know in the comments below:

Did the weekend take you by surprise?
Or you were fully ready for it with task list completed?
What is on next week's task list?
If you're planning to join New Relic or PagerDuty at one of the events above, which one do you have in mind?

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 13 Feb 2023 20:52:22 +0000

It's a new week, which means: however the last week ended we get a fresh start to all our plans and dreams. Because sometimes, you just gotta hit restart and try again.

What will inspire you to such great heights this week? I can't wait to find out!

A warm welcome 👋 to all our new members!

Thank you for stopping by to say hello @solomedianet! If you're an SRE in our community, please feel free to let Solomon know in the comment thread here 🤗

ICYMI

@oleonardorodrigues returned with 2 fresh posts! Find out how to use terraform-docs to create automatic docs for your Terraform modules and how Leonardo uses Dependabot to update dependencies in GitHub 💪
@bhushan_pathak took us on a trip through the CKA exam experience with plenty helpful tips if you're planning to take yours anytime soon. (Just a reminder, it's ok if you're not) 🍀
@eriklz reviewed TidyCloud's predictions for 2022 and updates their podcast recommendations https://community.ops.io/eriklz/tidy-cloud-aws-issue-41-buildpacks-media-recommendations-2022-review-1hmo 🔮
@dejanualex showcased the security of Platform One's Iron Bank 🔐
Meanwhile, conference season is in full force for the PD team. Good luck with the jet lag, y'all! You are tougher humans than me, that's for sure 😰
DevInterrupted shared their latest podcast episode with an invitation to LinearB's event on Wednesday 15 February: Scaling Developer Efficiency in 2023
Using metrics & automation to execute your R&D strategy in a down market!
Speaking of events, New Relic's show is on the road. Meet the New Relic team at FutureStack San Francisco on Wednesday 15 March.

If you're nowhere near the Bay Area, maybe O11y Day: Miami on 23 February and FutureStack São Paulo on 8 March are closer to your doorstep?

What are you planning for the week ahead?

👉 Let me and the community know by commenting below. I'll be checking back with you next week to see how it went!

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 06 Feb 2023 19:16:29 +0000

A warm welcome 👋 to all our new members!

@jgaskins published their first post detailing how they used cert-manager to configure the CA certificates for Mutating Webhooks 🥳

From our Welcome post, please take a moment to say hello to @nicedamsel25 (thread here), @aerosouund (thread here), and @bmayhew (thread here) - we're delighted to meet you all! 🤗

ICYMI

@the_cozma troubleshooted a pod stuck in CreateContainerConfigError in Kubernetes and took time to show us how it works,
@techielass showed us how to deploy an Azure Resource Group with Terraform,
@lnxchk was busy with a tutorial on using PagerDuty's App Event Transformer for GitHub repo alerts and made sure to deliver PD's trusty weekly update post 💪
I would be remiss if I didn't highlight and celebrate @techwatching's first non-Pulumi post!! ✨
@alvaradodaniel3 shared the announcement that New Relic Vulnerability Management is now generally available with an early-adopter rate - now's a great time to check it out!

Finally, I want to take a moment to thank @tylerauerbeck for looking out for this community by quietly reporting spam content. Thanks, Tyler, for being proactive and consistent. 🙏

What are you planning for the week ahead?

👉 Let me and the community know by commenting below. I'll be checking back with you next week to see how it went!

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 30 Jan 2023 19:59:00 +0000

I realized today that I ask y'all every week what your goals are, and never really share mine. Part of that's because the glamorous life of a community moderator is kinda-sorta like your friendly neighbourhood janitor (either/or neighbourhood street signs 😅).

I say that to say, if there's anything you're curious about here, you can always ask a mod. Questions that might come to mind are,

"How do I get a badge?",
"Where did the unicorn emoji go?",
"How do I schedule a post?",
"Why aren't my gifs displaying as expected?". As @techwatching has discovered: sometimes we have answers, sometimes we don't, but we will always try to help you find understanding!

So, don't be shy! Ask a mod a question today!

A warm welcome 👋 all our new members!

ICYMI

@patrick_londa outlined the steps to migrate from AWS EC2 launch configurations to launch templates
@eriklz went KonMari on devops, which resulted in more focus on Pulumi ✨
@techwatching showed us how to provision an Azure SQL Database with Active Directory authentication, using Pulumi whilst keeping the heat on Terraform (and staying true to the position Alexandre first announced to us in this post 🌶
@alvaradodaniel3 shared the latest New Relic news](https://community.ops.io/newrelic/solve-problems-faster-communicate-alongside-your-telemetry-data-with-slack-17o4)
@expoverse took it back to basics with a tutorial for a simple webpage + CI/CD pipeline 🎉
@eyalestrin continued the Cloud Native Applications series with this overview for securing cloud-native applications 🔐
@techielass created this guide to using GitHub Actions Secrets to securely store repository/environment/organization secrets and use workflows to call them for Azure deployment🤫
and where do I even begin with @danielepolencic's Learn Kubernetes Digest? It is absolutely crammed full of articles, tools, tutorials, calls for papers, events, jobs, and more! 🔥

What are you planning for the week ahead?

👉 Let me and the community know by commenting below. I'll be checking back with you next week to see how it went!

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 23 Jan 2023 21:00:18 +0000

Welcome back to everyone returning after the holidays and Happy Lunar New Year to everyone who was celebrating over the weekend 🐇

We had a lot of fresh content posted last week, so I'll do my best to pick out some themes, but please forgive any oversight on my part!

A warm welcome 👋

to @devinterrupted and @jit_mvs who joined us recently 🤗 - we encourage you to share a bit more about yourself in the Welcome Thread when you get a chance.

ICYMI

Final call for the webinar on CloudOps and Cybersecurity Predictions for 2023 tomorrow, 24 January 24 at 10AM PST, with @haviv and special guest Sarbjeet Johal.

Register on the BlinkOps website here. This event will be hosted by the Blink team, and they're available to answer any questions you may have in the comments below.

Let's kick off this week's round-up with a review of the wealth of Kubernetes content our community has blessed us with:

@danielepolencic shared not one, but two K8s guides: this hands-on tutorial for building an ingress controller in bash and an explainer on why you can't ping a Kubernetes service 😎
Meanwhile, @argonaut's double-headliner included a primer on Managing Kubernetes Secrets,
and @the_cozma is back with this breakdown of a couple of security benchmark monitoring tools for Kubernetes 🥳

We also gorged on AWS and AWS-adjacent content:

courtesy of our new friends @jit_io, who shared this runthrough of how they fixed their InvalidIdentityToken error 💊
@eyalestrin outlined strategies to mitigate being caught out by a lack of cloud resources , and followed this up with the first part of a new series on cloud native apps.
@techielass rounded it out with a tutorial on installing Terraform in the AWS CloudShell ✨

Phew! And there was so much more, ops friends...

Another post from @techielass reviewed six common CI/CD tools - your preferred tool didn't make the cut? Join the discussion in the comments on Sarah's post!
@techwatching is really leaning on us for that Pulumi badge with this consistent content! This week, Alexandre walked us through using Pulumi Watch for hot reload 💫
and @lnxchk introduced us to chaos engineering practices while sharing the PagerDuty team's Rundeck Community Survey with the promise of prizes 🎊

What are you planning for the week ahead?

👉 Let me and the community know by commenting below. I'll be checking back with you next week to see how it went!

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 16 Jan 2023 20:02:53 +0000

One of the things I really appreciate about this community is the varied levels of the content and discussions that take place.

We have community members across all stages of their careers who are sharing articles and guides for readers at all levels. It makes for a thrilling blend of content that informs and inspires!

Thank you to everyone who shares their insights and high-level ideas, or takes the time to write up tutorials and overviews of concepts that they found useful when they were starting out.

I want to give a special thanks to the people asking questions and those taking the time to answer them:
A welcoming community for DevOps practitioners of all levels needs us to ask the questions we don't yet know the answers to - and to explain how we learned to answer the ones we do.

A warm welcome 👋

to @bhausabp who stopped by with an introduction in the Welcome Thread. Please join me in greeting Bhausab 😊

ICYMI

On 24 January 24 at 10AM PST, the Blink team will be presenting a webinar on CloudOps and Cybersecurity Predictions for 2023 with @haviv and special guest Sarbjeet Johal.

Register on the BlinkOps website here, and drop a comment for @johnson_brad with any questions you may have.

This week has eased us into the new year with plenty of beginner-friendly content:

@jatin published a beginner's guide to build tools,
and @priyanshisharma shared a DevOps career roadmap,
while @techielass put together this intro to Terraform 😎
Meanwhile @techwatching is conspiring with @eriklz to persuade us to create a Pulumi badge with yet more Pulumi content, this time in tandem with Azure - let me know what you think about the Pulumi badge below 😉
speaking of Erik, we started a fun discussion about the origins of industry reports on this latest Tidy Cloud bulletin and would love to hear your thoughts! 🤔
@lnxchk followed up this post on service objects with a follow-on on splitting escalation policies. Mandi and the PD team are also back on the road, so follow their brand page to get updates on where they'll be heading next! ✈
The New Relic team reminded us about an upcoming update to their TLS requirements for all inbound connections and introduced us to their partner, Hack the Hood.

What are you planning for the week ahead?

👉 Let me and the community know by commenting below. I'll be checking back with you next week to see how it went!

What do you hope to achieve this week?

Ella (she/her/elle) — Mon, 09 Jan 2023 18:51:50 +0000

We are already in Week 2 of 2023. What changes have you already noticed - whether in your own mindset, your team dynamics at work, the news cycle, or right here in The Ops Community?

No prizes for noticing the unicorn reaction is missing, but notable mention to @eriklz for reading my post about it (thanks for noticing, Erik!) 😂

A warm welcome 👋

to @iswazilamanyamande, @uchelouis45, and @abhinavd26, who all introduced themselves in the Welcome Thread. Stop by to say "hello" back to them if you find a moment 😊

ICYMI

We welcomed @galco and the Firefly team to The Ops Community with this post about their event-driven architecture involving DynamoDB and Terraform 🙌
If you're trying to decide between Pulumi or Terraform for your next IaC solution, @techwatching has written the in-depth breakdown you're looking for, and would love to know what you decide!
@briancaffey adds CDK to the comparison table in this experiment, deploying the same web app in 3 different libraries - join the rest of the conversation here 🤩
@danielepolencic is back with more K8s tips, this time on debugging a pod in production. Check out the debugging flowchart for help with more Kubernetes troubleshooting 😎
@aowendev is keeping up the content cadence with this guide to manually migrating from Forestry to its newest incarnation: TinaCMS - thanks for sharing your migration process with us, Andrew! 🦙
@jatin is back with the latest instalment of the Build Better CI CD Pipelines series; this tutorial explores AWS CodePipeline by building a CD pipeline with s3 - let Jatin know how you get on!
Our friends at PagerDuty shared their first weekly update of 2023 and guess who's in it? That'll be us! 🥳

I joined @johnson_brad to chat with @lnxchk for the latest episode of Page It To The Limit, and had a great time talking about all y'all!

What are you planning for the week ahead?

👉 Let me and the community know by commenting below. I'll be checking back with you next week to see how it went!

Feature update: Retiring the unicorn reaction 🦄

Ella (she/her/elle) — Tue, 03 Jan 2023 20:17:31 +0000

New year, new reaction buttons - or at least, that's what we're hoping for!

We have decided to follow in the footsteps of DEV, by giving the unicorn reaction button a well-deserved rest. RIP 🦄 friend.

This was in part guided by the Forem team's awareness that users have to scroll through the entire post to reach the comments section, and in part because people have been confused about the unicorn reaction since time immemorial. They resolved both issues by replacing the unicorn for a jump-to-comments button, as you can see in the sidebar of this post and in the screenshots below.

Before:

After:

Note how the new reaction buttons have a comments button in place of the unicorn, which will jump you directly to the comment section. 🗨

As explained in the DEV post, there will eventually be more reaction buttons to replace the unicorn, but for the time being we are moving into 2023 with the comments button...

What do you think?

🦄 Will you miss the unicorn?
🗨 Will this will make it easier to comment on posts?
🗨🦄 Why not both?