The Ops Community ⚙️: Patrick Londa

Aligning Your AWS Account with the FFIEC Cybersecurity Standards

Patrick Londa — Wed, 05 Jul 2023 20:45:07 +0000

Companies in the banking and finance industry must adhere to high security standards since they are high-value targets for bad actors.

Industry-specific organizations like the Federal Financial Institutions Examination Council (FFIEC) have established guidelines to help companies ensure compliance with applicable laws and regulations.

In this guide, we’ll show you how to check if your AWS account adheres to the cybersecurity standards set forth by the FFIEC using automations in Blink.

Understanding the FFIEC Cybersecurity Standards

Established in 1979, the Federal Financial Institutions Examination Council (FFIEC) is a U.S. government interagency body of five organizations working together to ensure the safety and soundness of the banking system.

The FFIEC coordinates common standards for banks and develops uniform guidelines and examinations for all financial institutions. It also releases tooling, like the Cybersecurity Assessment Tool (CAT), to help financial institutions evaluate their cybersecurity risk and develop appropriate controls. The CAT is a document that provides a framework and guidance, but it does not interactively assess an AWS account for compliance.

FFIEC Cybersecurity Guidance for AWS

An audit of an organization's AWS environment is a critical part of FFIEC compliance requirements. AWS provides the tools and services necessary for financial institutions to adhere to FFIEC regulations, but each organization must ensure that its environment meets the specific requirements of the FFIEC.

AWS provides operational best practices for FFIEC compliance, including a list of control IDs, AWS configuration rules, and guidance.

Here are some examples of controls that organizations using AWS must follow to meet the FFIEC guidelines:

An inventory of organizational assets is maintained.
An information security and business continuity risk management function exists within the institution.
The risk assessment identifies internet-based systems and high-risk transactions that warrant additional authentication controls.
Information security threats are gathered and shared with applicable internal employees.
Audit log records and other security event logs are reviewed and retained in a secure manager.

For each of these controls, there are a few to several configuration rules in AWS that could apply to your organization, depending on the guidance.

Manually checking whether your EC2 volumes are all encrypted, your IP addresses are all private, or you have the right password policy in place could take days or weeks.

If you want to check your AWS environment for compliance quickly, you can use automation to get a comprehensive report based on these controls.

Automating FFIEC Compliance for AWS with Blink

With one automation in Blink, you could quickly scan your AWS environment to check your FFIEC compliance against the controls and generate reports with the findings.

Blink Automation: Federal Financial Institutions Examination Council Compliance Report for AWS

When this automation runs, it executes the following steps:

Generates a Cyber Risk Management and Oversight Report.
Generates a Threat Intelligence and Collaboration Report.
Generates a Cybersecurity Controls Report.
Generates an External Dependency Management Report.
Generates a Cyber Incident Management and Resilience Report.
Sends Report results to a specified email.

You could set this automation to run weekly, monthly, or quarterly so you can validate that you are maintaining your compliance over time.

You may also have other compliance checks you need to run beyond this one with the Financial Federation Institutions Examination Council guidelines. What about SOC, ISO, or PCI compliance?

There are over 7K pre-built automations in the Blink Library that make it easy to gauge your environments against industry standards.

To start streamlining your compliance and security checks today, you can get started by signing up for a free trial or guided demo of Blink.

Introducing Blink Copilot: Generative AI for Security Workflows

Patrick Londa — Wed, 31 May 2023 19:27:58 +0000

Today, we’re announcing Blink Copilot, the first ever generative AI for automating security and IT operations workflows. This innovative tool makes it possible for any security operator to generate no-code workflows using a simple prompt.

When Blink was founded in 2021, we knew security teams needed a better way to automate internal workflows. Low-code platforms had already transformed business operations for CRM and marketing, and it was only a matter of time before low-code solutions enabled security teams to automate their own workflows across their stacks.

Recent advancements in large language models (LLM) and generative AI have supercharged our mission, finally making true no-code automation possible. That’s why today, we’re excited to announce Blink Copilot, the first ever generative AI for security workflow automation.

Blink Copilot enables security teams to generate any simple or complex workflow, instantly. Generative AI makes it possible to build workflows without writing code or needing to be an expert in target applications.

The Critical Need for Security Automation

According to the 2022 (ISC)² Cybersecurity Workforce Study, there are over 3.4 million open security jobs and too few skilled security operators to fill them. Meanwhile, cybersecurity breaches in 2022 were more costly than ever before.

With such high stakes and a massive skills gap, security automation is necessary for organizations to defend against the massive volume of attacks enterprises face on a daily basis.

Blink Copilot empowers teams of any size to build any no-code, low-code, or code security automation using generative AI. With Blink Copilot, security and IT operators of all skill levels can now leverage AI to increase productivity and deliver workflows to help protect their organizations.

Blink Copilot unlocks unparalleled automation capabilities for security teams. Security automation projects that once took months are now built in seconds using Blink Copilot.

Automate Security Beyond the SOC

Blink Copilot empowers security teams to automate any operations workflow, effortlessly. Security practitioners regardless of skill set can automate any operational task or security workflow using a simple text prompt.

Blink Copilot can help security teams automate workflows for:

SOC & Incident Response
Cloud Security
IT & SaaS Security
Identity & Access Management
Governance, Risk & Compliance

Using Blink Copilot, security teams can finally automate workflows beyond the SOC. Blink customers get out-of-the-box automations for popular security use cases and powerful automation capabilities backed by the robust Blink platform.

Our internal team at Blink Ops has already generated over 7,000 workflow automations. Now with Blink Copilot, we’re empowering any team to build no-code, low-code, or code security workflow automations using generative AI.

Powerful No-Code Security Automation

Blink Copilot is backed by the most powerful no-code security automation platform ever built.

Key features of the Blink platform include:

Blink Copilot: The first ever generative AI for automating security and IT operations workflows
Low-Code Editor: Drag-and-drop user interface that makes it easy for security teams to customize workflow automations
Automation Library: 7,000+ security workflow automations built by the Blink community
Self-Service Portal: Shift-left service requests by making automated workflows available in a self-service portal or interactive Slack/Microsoft Teams app

Blink was designed for enterprise security teams, offering robust platform features like secure workspaces, on-prem runners, and multi-tenant support for MSPs and MSSPs.

Blink is committed to upholding the highest grade of industry security standards, including SOC 2 Type 2, GDPR, ISO 27001 compliance. Blink can also help teams leverage no-code automation to achieve compliance across their own organizations.

Learn more about the Blink platform and technical capabilities in the Blink documentation.

Get Started with Blink Copilot

Blink Copilot completely transforms how security teams will automate simple and complex workflows.

Security teams can take advantage of Blink Copilot today to shift-left internal workflows. Along with 7,000 out-of-the-box automations for common security use cases and powerful automation capabilities backed by the robust Blink platform, enterprise security teams can finally achieve operational excellence across their entire stack.

For more information on Blink Copilot, visit the Blink Ops website.

How to Find and Update Public EC2 AMIs in AWS

Patrick Londa — Fri, 05 May 2023 19:30:54 +0000

Resources in AWS should only be accessible to users who require them to complete tasks. If this principle of least privilege isn't followed, you increase the risk for data leakages or unauthorized access.

If you have EC2 AMIs that are publicly-shared for example, they are available to any AWS account and may contain sensitive data about your organization such as passwords, SSH keys, and configuration details.

In this guide, we will explain how to find and restrict publicly-shared AWS EC2 AMIs so you can ensure that sensitive information about your applications is not exposed.

Understanding AMIs for AWS EC2

An Amazon Machine Image (AMIs) is an image that enables you to easily launch Amazon Elastic Compute Cloud (EC2) instances with the necessary requirements preconfigured.

Each AMI includes instructions for block device mapping, launch permissions, and a root device volume, either backed by the EC2 instance store or at least one Amazon Elastic Block Store (EBS) snapshot.

You can use an AMI to launch single instances or multiple servers in a cluster. For example, if you are launching a web server that handles a large volume of traffic, you can use an AMI to deploy it quickly. You can also launch multiple instances in the same cluster and configure them to work together.

These AMIs can be shared with specific users, giving them access to the same configurations and settings. If they are publicly-shared, AMIs can reveal sensitive information about how your instances function.

How To Find Any Public AMIs in AWS

Since AWS EC2 AMIs are publicly shared by default, it's easy to miss if you haven't taken the time to check your settings. Luckily, there are two ways to check whether your AMIs are publicly shared. You can use the Amazon EC2 Console or the AWS CLI, depending on your AWS environment.

Using the AWS Console:

This method is the most straightforward way to check if your AMIs are publicly shared. Follow these steps:

Log in to the AWS Management Console.
Select EC2 from the list of services.
Click AMIs in the left sidebar menu under the IMAGES section.
To see all the public AMIs, use the first filter in the AMI list and choose Public Images.
To check individual AMIs, select one and scroll down to the Permissions section and check the current permissions. If the selected AMI is publicly shared, the EC2 dashboard will display the following message, "This image is currently Public" in the Permissions section.

Using the AWS CLI:

You can also use the AWS Command Line Interface (CLI) to check whether your AMIs are publicly shared. This process is a bit more complex, but here's how it works:

Run the following describe-images command:

aws ec2 describe-images --executable-users all

The command will return a list of all the publicly shared AMIs in your account. If the command returns images with the Public parameter value of "true" then the AMI is publicly accessible.

For instance, the AMI in question is publicly shared if the output is something like this:

{"Images": {"ImageId": "ami-1234abcd", "Public": true } }

Now that you have identified public AMIs, next you can update them to make them more secure.

Changing Public AMIs to Private AMIs in AWS

You can update public AMIs to restrict access using either the AWS Console and CLI.

Using the AWS Console:

Log in to the AWS Management Console again and select EC2 from the services list.
Select the AMI you want to make private from the list of AMIs in the IMAGES section.
Head to the Permissions tab from the bottom bar menu on your dashboard. Click on Edit AMI Permissions.
Then select Private and Save changes to make the AMI private. The Permissions tab will now display the following message, "This AMI is not shared with any other accounts."

Using the AWS CLI:

Run the following modify-image-attribute command to make an AMI private:

aws ec2 modify-image-attribute 
    --image-id ami-1234abcd 
    --launch-permission "{\"Remove\":[{\"Group\":\"all\"}]}"

This will immediately make the AMI private while preserving all its existing permissions.

To double-check, you can list the attributes of the image by running the following describe-images command for the AMI with the id “ami-1234abcd”:

aws ec2 describe-images --image-ids ami-1234abcd

The output should now display the Public parameter as false as shown here:

{"Images": { "ImageId": "ami-1234abcd", "Public": false}}.

Now, you have successfully secured your non-compliant AMI.

Automating AWS Permission Checks for EC2 AMIs with Blink

It isn’t hard to find and secure public AMIs, but it can be time-consuming if you have many that need to be addressed or you want to check this on a regular basis.

How do you ensure that you don’t have overly-permissive AMIs at any given time? If you are only doing this task with CLI commands or steps in the console, you will need to context-switch to oversee each step.

With an automation platform like Blink, you can run this automation to identify and notify your team of new public AMIs so you can maintain consistent compliance.

Blink Automation: Ensure EC2 AMIs are Not Shared Publicly in AWS

This automation in the Blink library executes the following steps:

Checks if there are any publicly-shared AWS EC2 AMIs.
Sends a report to a specified email address.

This is a simple automation, which makes it easy to customize for your organization’s needs. For example, you can add actions to update the settings of non-compliant AMIs with just an approval via Slack.

This automation and 5K more are available for you to use right away from the Blink library. Instead of manually implementing best practices with the AWS console or CLI, these ready-to-use automations enable you to set and enforce policies across your organization or teams.

You can also build custom automations to match your organization’s unique use cases, whether you want to schedule a task, run steps based on an event in a certain tool, or share a self-service application for your coworkers to use.

Start a free trial of Blink today or schedule time with us to see more.

How to Search for an IOC Across Devices in CrowdStrike

Patrick Londa — Sun, 09 Apr 2023 16:39:43 +0000

When malware is detected on one of your organization’s devices, it will have characteristics called Indicators of Compromise (IOCs), such as certain hash values, urls, or IP addresses.

You can use these IOCs to look across your organization’s devices to identify lateral movement associated with an attack.

In this guide, we’ll show you how to use CrowdStrike to detect if IOCs associated with malware are present on any other devices at your organization.

Searching for an IOC Across CrowdStrike Hosts

You can search for IOCs on other devices either by using the CrowdStrike Console or by using the CrowdStrike API.

Using the CrowdStrike Console:

1: First log in to the CrowdStrike Falcon Console.

2: Open the left-hand menu and select Investigate.

3: Depending on your IOC type, choose the related link under the Search section. For example, if you are looking for an IOC that is a domain, you can choose Bulk domains.

4: Input your IOC value and specify the time range you care about. Then click Submit.

In the results, you’ll see which hosts have observed the IOCs you’re investigating and you’ll see details on the process-level as well.

You may want to contain any additional hosts now associated with the IOC. We wrote a guide on containing hosts here. For your audit trail, you can export this data by hovering over either section and clicking the Export icon.

Using the CrowdStrike Falcon API:

The platform also offers an API which allows administrators to easily programmatically manage their sensors. You can use the endpoint that geographically aligns with your specific CrowdStrike account:

US-1 “api.crowdstrike.com”
US-2 “api.us-2.crowdstrike.com”
US-GOV-1 “api.laggar.gcw.crowdstrike.com”
EU-1 “api.eu-1.crowdstrike.com”

In the examples we show later, we’ll use “api.us-2.crowdstrike.com”.

CrowdStrike’s API documentation is available after you log in here, and you’ll see information about how to use OAuth2 for authenticating your requests.

Before you start, you need to make an access token request, including your client ID and client secret. You’ll get an access token in response that will be valid for 30 minutes after that. The API calls you make after that initial call will include that token.

Next, make a GET request to this endpoint with IOC type and value specified:

https://api.us-2.crowdstrike.com/indicators/queries/devices/v1?type=sha256&value=XYZ

You can use the following IOC types:

sha256
md5
domain
ipv4
ipv6

You can also use the parameters limit and offset to manage pagination of results. With the response, you’ll be able to see all the resources that have observed the specific IOC.

Automatically Searching Across Devices for IOCs with Blink:

Checking if similar IOCs exist on other devices at your organization is one of many steps in responding to a security alert. While it isn’t difficult to do, it takes time and context-switching.

With Blink, you can handle this task automatically by just inputting an IOC type and value.

With these IOC inputs, this automation runs the following steps:

Searches for other devices that have observed the IOC.
Parses the results to format them into an easily readable report.
Sends the report to the relevant SecOps team member via Slack.

This is a simple automation, and that makes it easy to customize. For example, you can add this as a subflow in larger incident response automation triggered by a CrowdStrike alert.

You can also use any of the 5K automations in the Blink library, or build new ones from scratch to fit your unique needs. In Blink, there are hundreds of native integrations and thousands of actions available to make building easy.

Start a free trial of Blink today to see how easy automation can be.

How to Get User Activity From Your Azure Logs

Patrick Londa — Thu, 23 Mar 2023 18:14:47 +0000

It’s important to be able to audit user activity in Azure, whether you are dealing with a security incident or just want to fully review the actions a user has taken.

If one of your developers has their account compromised, reviewing their user activity can be a necessary task to ensure that they haven’t done anything malicious to your Azure account and resources, or exfiltrated data like access keys or secrets.

In this guide, we’ll show you how to retrieve all the activity logs for a given user in Azure to quickly assess the scope of the threat.

How to Get User Activity From Azure Logs

Azure Monitor collects and organizes all log and performance data from Azure resources, and you can access the activity logs for the last 90 days through steps in the console or CLI commands.

Using the Azure Monitor Log:

1. Open the Azure console, and navigate to the Activity log view. You can do this either by clicking into a specific resource or by searching for Azure Monitor to see all activity logs across your account.

Source: Azure documentation

2. Next, you can add the Events initiated by filter to view only the activity logs related to a specific user.
3. You can also adjust the timespan filter to specify the full duration you care about. You can also select Edit columns if there is additional information you want to view.
4. To export the logs, click on Download as CSV.

This method is relatively simple, but it does require logging in to the console and manually working through the steps. Let’s look at running this same search from the Azure CLI.

Using the Azure CLI:

If you aren’t already set up with the Azure CLI, you’ll need to install it locally. From there, you can run the az monitor activity-log list command to list and query activity logs.

Here is the format to get the activity logs for a specific user:

az monitor activity-log list --caller [USER-NAME]  
--offset [TIME-RANGE]

The offset parameter is formatted as XXdXXh, and defaults to 6h if not specified.

Here’s an example command to get the activity logs for John Smith:

az monitor activity-log list --caller john.smith@blinkops.com  
--offset 14d

You can these additional parameters to modify the results:

--start-time or --end-time: use the format of date (yyyy-mm-dd), time (hh:mm:ss.xxxxx), and timezone (+/-hh:mm).
--resource-group, --resource-id, --namespace: use this to narrow to only activities affecting those specific resource areas.
--max-events: use to change the limit of the results, the default is 50

You can see all of the options in the command documentation.

This approach is more easily repeatable than the console steps, but it requires you to remember the commands and be familiar with the Azure CLI.

With a no-code automation tool like Blink, you can run this task effortlessly.

Getting User Activity from Azure Faster with Blink

Retrieving user logs manually and adding them to a ticket could be a waste of valuable time, especially if your team is responding to a security incident.

With Blink, an automation can be triggered to pull and enrich Azure activity logs and other information for a compromised user right away.

Blink Automation: Get User Activity from Azure Logs

The automation shown above is in the Blink library and is set up as a self-service app – where a team member can specify input parameters and get all the activity logs sent to an email address.

When it runs, it executes the following steps:

1. Fetches the logs from Azure monitor for a specified email address.
2. Formats the logs for better readability.
3. Reports the results to the specified email address.

If you’re handling a security incident, you can have this flow as part of an automation that is triggered by a malware or DLP alert for a given user. That way, you have all the logs and information you need to assess the risk for your organization.

You can import this automation from the library into your account and customize it based on your organization’s needs. For example, you can drag-and-drop new actions into the canvas or set up conditional subflows.

You can build your own automation from scratch or use one of our 5K pre-built automations today.

Start a free trial of Blink today or schedule time with us to see more.

How to Verify IOCs and Enrich Your Incident Response with VirusTotal

Patrick Londa — Mon, 06 Mar 2023 19:03:01 +0000

IOCs, or Indicators of Compromise, are pieces of information that can be used to detect malicious activity on a network or system. By using a threat intelligence tool like VirusTotal, organizations can get more information about IOCs like the type of threat detected, its origin, or even which antivirus engines are detecting it.

In this guide, we'll show you how to use VirusTotal to verify an IOC and incorporate its results into an automated incident response workflow.

Verifying IOCs Using VirusTotal

When endpoint detection tools like CrowdStrike or SentinelOne detect malware, they will alert organizations and the incident will list IOCs. IOCs include such data as the IP address or domain name of a suspicious website, hashes associated with malware, and various other details related to an attack.

Using VirusTotal on a Browser:

If you are a SOC analyst and you want to know more information about the malicious website or software, you can go to the VirusTotal website. VirusTotal is a free service offered by Google which allows users to upload files, enter URLs, search IP addresses or file hashes, and scan these against the platform's antivirus databases.

Here’s an example of what you would see if you submitted a file hash value that isn’t malicious:

If you have a malicious file hash, you’ll see this information:

You’ll be able to view details like creation time, when it was first submitted, and when it was last analyzed, how it behaves, community comments, and more.

Using the VirusTotal API:

The VirusTotal API allows users to integrate their own custom scripts into the platform, enabling more advanced incident response workflows. For example, a script can be used to automatically upload suspicious files for scanning against the platform's databases.

To access the VirusTotal API, you’ll need to create an account and then you’ll have a free public API key you can use.

You can use this endpoint to get a file report:

curl --request GET \
--url https://www.virustotal.com/api/v3/files/{id} \
--header 'x-apikey: <your-API-key>'

You can use this endpoint to get a URL analysis report:

curl --request GET \
--url https://www.virustotal.com/api/v3/urls/{id} \
--header 'x-apikey: <your-API-key>'

And you can use this endpoint to get an IP address report:

curl --request GET \
--url https://www.virustotal.com/api/v3/ip_addresses/{ip} \
--header 'x-apikey: <your-API-key>'

If you review the API documentation, you may see other popular endpoints that better map to your incident & response process.

In the context of responding to an alert, you could save yourself a step by setting up a script to automatically call these endpoints with each respective piece of information and enrich your alerts.

Taking Action Based on VirusTotal Reports with Blink

When you enrich alerts with VirusTotal reports, you can save you time manually reviewing each detail. Instead of stopping at enrichment, what if you could utilize the results to kick off conditional responses?

For example, if an IOC is verified and has a high threat score, should you:

Immediately contain that device using your endpoint detection tool?
Search other devices across your organization to see if that IOC is present?
Add the IOC to your firewall and EDR blocking rules?

These might be the right steps, but if you scripted them all, you might not be able to control your automations easily at each stage.

With Blink, you can build event-based automations to facilitate incident response with conditional workflows and approval steps along the way. When a new alert is raised in an EDR like CrowdStrike or SentinelOne, you could immediately enrich that alert with reports from VirusTotal on each IOC.

Depending on the threat level, you can speed up your time-to-action with conditional response steps to fully leverage the combined power of the security and communication tools you’re already using.

You can start building a better response process with this pre-built Blink automation that verifies IOCs with VirusTotal. We have over 5K ready-to-use automations just like it in our library. By combining and customizing them, you’ll be able to create the workflows that match each security situation.

Get started with a free trial of Blink today or schedule time to see a demo.

How to Migrate from AWS EC2 Launch Configurations to Launch Templates

Patrick Londa — Tue, 24 Jan 2023 22:22:15 +0000

For EC2 Auto Scaling, AWS is currently urging all customers to switch from using launch configurations to launch templates instead.

As part of this push, starting in 2023, launch configurations do not support any new Amazon EC2 instance types released after Dec. 31st, 2022. In practice, AWS could release a more cost-effective instance type that better fits your use case, but you won’t be able to take advantage of it until you migrate from launch configurations to launch templates.

In this guide, we’ll explain the basics of launch configurations and launch templates, and show the steps for migrating them over.

Launch Configurations vs. Launch Templates

Amazon EC2 Auto Scaling groups need information to guide how to launch new EC2 instances when they scale. They rely on one source of truth; either an EC2 instance (automatically converted to a launch configuration), a specified launch configuration, or a launch template.

Launch Configurations

Launch configurations are instance configuration settings utilized by an EC2 Auto Scaling group to inform the new EC2 instances it launches. You must specify standard information when setting up a launch configuration, including the following:

ID of the Amazon Machine Image (AMI)
Instance Type
Key Pair
Security Group(s)
Block Device Mapping

For inspiration, you can use similar settings for previously launched EC2 instances if they match your use case. Once you create your launch configuration, you cannot make changes to it. You can only create a new one and update your Auto Scaling group to use the new one. If you do that, existing instances will not be immediately updated.

Launch Templates

Launch templates function similarly to launch configurations by specifying instance configuration information with similar information you would place in a launch configuration file. The major difference between launch configurations and launch templates is that you can set up multiple versions of a launch template.

Versioning launch templates lets you establish a subset of the complete parameter set. You can reuse these to set up other versions of the base launch template. For example, you can create a launch template with a defined base configuration without including an AMI or user data script.

After creating the launch template, you can add a new version with the AMI and user script for testing. You end up with two versions of the template. That means you can always have a base configuration for reference, then create new template versions as needed. It’s also possible to delete test template versions when they are no longer necessary.

Launch templates are recommended because you can access the latest improvements. Some Amazon EC2 Auto scaling features aren’t available when you use launch configurations. Launch templates also let you use newer generation features of Amazon EC2.

While parameters in launch templates are optional, you can’t add an AMI to one if you do not specify it when creating an Auto Scaling group. If you specify an AMI but leave out the instance type, it’s possible to add one or more when setting up your Auto Scaling group.

How to Migrate Launch Configurations to Launch Templates

Copying launch configurations to convert them to launch templates, you will need to use the AWS Console.

Migrating One Launch Configuration

Anyone currently using launch configurations can migrate them over to launch templates by copying them into the console.

Open the EC2 console.
Look for Auto Scaling in the navigation pane, then choose Launch Configurations.
Choose the launch configuration to copy.
Select Copy to launch template, Copy selected to set up a new template with the same name and options as your configuration.
Add the name of your current launch configuration file or a new name in the New launch template name field. The name must be unique.
Select Copy.

Migrating All Launch Configurations‍

Follow the steps below if you wish to move all your launch configurations to launch templates in the console:

Open the EC2 console.
Look for Auto Scaling in the navigation pane, then choose Launch Configurations.
Choose the launch configuration to copy.
Select Copy to launch template, Copy all to copy all your configurations within the current Region to a new launch template named after your configuration.
Select Copy.

Updating Auto Scaling Groups to Use Launch Templates

Using the AWS Console:

Navigate to the AWS EC2 console, open the navigation pane, and select Auto Scaling Groups.
Click on the check box associated with the Auto Scaling group you want to update.
In the Details tab, choose Launch configuration, then click Edit.
Select Switch to launch template, then select the appropriate launch template.
You’ll need to select a Version for the launch template. You can customize whether the Auto Scaling group uses a default version when it scales out, or uses the latest version.
Click Update to apply the changes.

Using the AWS CLI:

You can run this update-auto-scaling-group command:

aws autoscaling update-auto-scaling-group
--auto-scaling-group-name <value>
--launch-template <value>

An Auto Scaling group cannot have both a launch configuration and launch template parameter set. To update your Auto Scaling group to use a launch template instead of a launch configuration, you can include a launch template value in this command, like in this example:

aws autoscaling update-auto-scaling-group
    --auto-scaling-group-name my-asg
    --launch-template LaunchTemplateName=my-template-for-auto-scaling,Version='2'

Once you’ve migrated your Auto Scaling groups to use launch templates instead of launch configurations, you should stop creating new launch configurations and only create launch templates and new versions.

Replacing Launch Configurations Automatically with Blink

If you have several Auto Scaling groups, making these updates can be manual and time-consuming. You also can’t ensure that new launch configurations are not created and utilized in your AWS EC2 Auto Scaling groups.

With Blink, you can use a no-code automation to quickly identify Auto Scaling groups that are using launch configurations, convert those launch configurations to launch templates, and update the groups accordingly. You can also set up checks to detect new launch configurations and notify owners that they should use launch templates instead.

Create your free Blink account and migrate fully to launch templates today.

How to Enable Autoscaling for a GKE Cluster

Patrick Londa — Mon, 12 Dec 2022 22:01:40 +0000

Autoscaling is an automated, node provisioning process that scales your GKE clusters depending on their workload needs. As a result, GKE clusters with autoscaling enabled scale up their node pool to offer more workload availability when demand is high and scale down their node pool to save on costs when demand is low.

You can control your cluster’s autoscaling by specifying a minimum and maximum number of nodes. You can also choose whether to use the default, balanced autoscaling method or the optimize-utilization setting.

In this post, we’ll walk you through the basics of GKE cluster autoscaling and show you how to enable it for your node pools.

Understanding Your GKE Autoscaling Options

When you enable autoscaling, you have the ability to set guardrails and preferences:

Minimum and Maximum Nodes

One of the decisions you need to make is the minimum and maximum number of nodes, either per zone (minimum nodes, maximum nodes) or in total (total minimum nodes, total maximum nodes) across your node pools.

For a minimum, you’ll always need at least 1 node for each zone the node pool is in. The autoscaler will never scale down to zero because at least 1 node is needed to run the system Pods.

When setting a maximum, you might want to consider the implications of a dramatic scale up. For example, if you scale beyond the IP address space you have allocated, you will receive an error and no longer be able to add new nodes. Consider these types of dependencies when selecting a maximum.

Balanced vs. Optimize-Utilization

There are two types of autoscaling profiles. The default profile is balanced, which means it scales up and down with a balance between availability of resources and node utilization. For example, balanced autoscaling allows for more nodes with lower utilization so that they are available if workloads increase.

The optimize-utilization autoscaling profile by comparison prioritizes concentrating utilization in fewer nodes, which enables the removal of underutilized nodes. The result is faster scale downs and lower resource costs. If you choose this profile, you may experience performance delays when new workloads require new resources to be provisioned. Depending on your performance requirements, optimize-utilization may be a useful way to lower your GKE operating costs.

Configuring Autoscaling for an Existing Node Pool

If you want to enable autoscaling, you can start by updating your existing node pools using GCP Console or the GCP CLI.

Using the GCP Console:

In the Google Cloud console, navigate to the Google Kubernetes Engine page.
Select the cluster you want to update from the displayed cluster list, and go to the Nodes tab.
Under Node Pools, select the node pool that you want to update and click Edit.
Under Size, check Enable autoscaling.
Specify values for Minimum number of nodes and Maximum number of nodes, and click Save.

Using the gCloud CLI:

You can use the following command to enable autoscaling for an existing node pool:

gcloud container clusters update CLUSTER_NAME \
    --enable-autoscaling \
    --autoscaling-profile=PROFILE \
    --node-pool=POOL_NAME \
    --min-nodes=MIN_NODES \
    --max-nodes=MAX_NODES \
    --region=COMPUTE_REGION

Plug in your details for the cluster, node pool, and region. If you only have one node pool, you can use default-pool as your value. The region value should be your Compute Engine region, or specific zone if it’s a zonal cluster.

The --enable-autoscaling flag invokes autoscaling. You can customize your configuration with --autoscaling-profile (balanced or optimize-utilization), --min-nodes, --max-nodes, --total-max-nodes, and --total-max-nodes.

Here’s an example of enabling autoscaling with an optimize-utilization profile for the pool-1 node pool of the demo-1 cluster:

gcloud container clusters update demo-1 \
    --enable-autoscaling \
    --autoscaling-profile=optimize-utilization \
    --node-pool=pool-1 \
    --min-nodes=1 \
    --max-nodes=4 \
    --region=us-central1

Enabling Autoscaling When Creating a New GKE Cluster or Node Pool

If you are creating new clusters or node pools, you can enabling GKE autoscaling by using settings in the GCP Console or GCP CLI flags:

Using the GCP Console:

In the Google Cloud console, go to the Google Kubernetes Engine page.
To set up a new cluster, click Create. To create a new node pool, select an existing cluster and click Add Node Pool.
Specify details for your cluster or node pool. For a new cluster, click default-pool under Node Pools from your navigation pane.
Next, you need to select the Enable autoscaling checkbox. For node pools, you’ll find it under the Size section.
Modify the values for Minimum number of nodes and Maximum number of nodes according to your requirements, and click Create.

Using the gCloud CLI:

When you are creating new clusters or new node pools, you can ensure that they have autoscaling enabled by including the --enable-autoscaling flag and specifying --min-nodes and --max-nodes values.

Here’s an example of creating a new cluster with this command:

gcloud container clusters create my-cluster --enable-autoscaling \
    --num-nodes=30 \
    --min-nodes=15 --max-nodes=50 \
    --region=us-central

Here’s an example of creating a new node pool with this command:

gcloud container node-pools create node-pool-1 
    --cluster=sample-cluster 
    --num-nodes=5
    --enable-autoscaling
    --min-nodes=5 --max-nodes=15

By updating your existing node pools, and creating new clusters and node pools with autoscaling enabled, you’ll be able to ensure your clusters are able to automatically adapt to changing workloads.

Checking Whether Autoscaling is Enabled with Blink

Autoscaling can make your GKE clusters more effective and efficient. If you want to make it a standard that your organization's GKE clusters have autoscaling enabled, then you can manually enable it using the steps above. Unfortunately, that requires many manual updates and doesn’t ensure that future clusters will also have autoscaling enabled.

With Blink, you can use no-code automations to quickly check if you have any GKE clusters without autoscaling enabled. You can then send a notification to Slack on a regular basis if there are any clusters missing this setting. You can be more agile with making adjustments and getting information when you are only a click away.

Create your free Blink account and better manage your GKE clusters today.

How to Manually Rotate Keys in GCP

Patrick Londa — Thu, 01 Dec 2022 17:26:54 +0000

Key rotation is a critical security practice. In GCP, you can either rotate keys by enabling automatic rotation or by rotating a key manually.

Manual rotations make sense if your key is compromised or if you are modifying your application to use a different or stronger algorithm.

In this guide, we’ll show you how to manually rotate keys using the GCP console and the gCloud CLI.

Manually Rotating Keys in GCP

You will need to have permissions granted by the Cloud KMS Admin role to rotate keys in GCP. If you want to also do the re-encryption step below, you’ll need permissions granted by the Cloud KMS CryptoKey Encrypter/Decrypter role.

Using the GCP Console:

These are the steps to manually rotate keys in the GCP Console:

Open the Key Management page from the Google Cloud Console.
Select the name of the key ring that contains the key you want to create a new version for.
Select the key for which you need to create a new version.
Click Rotate in the displayed header.
Again, click Rotate in the prompt to confirm the key rotation.

Now, you’ll see a new version of your key is created and is marked as the primary key.

If you want to use a different existing key version, you can make it primary key using these steps:

Choose the key whose primary version you want to update.
Click View More in the row of your intended key.
Select Make primary version in the menu.
In the confirmation prompt, click Make primary.

If you have encrypted anything with the prior key, you’ll need to re-encrypt it with your new key, and then destroy the old key. This encryption step can only be done with the CLI and we’ll show it in the encryption section below.

Using the gCloud CLI:

To run Cloud KMS on the command line, you’ll first need to install the latest version of gCloud CLI. Once you’ve done that, you can run this command:

gcloud kms keys versions create \
    --key <KEY_NAME> \
    --keyring <KEY_RING> \
    --location <LOCATION>

You can input values for each of these parameters:

<KEY_NAME> refers to the name of the key.
<KEY_RING> refers to the name of the key ring that consists of the key you want to rotate.
<LOCATION> refers to the key ring Cloud KMS location.

Here’s an example:

gcloud kms keys versions create
    --key=bowser
    --keyring=castle
    --location=global

You can then set an existing key version as the primary version with this command:

gcloud kms keys update <KEY_NAME> \
    --keyring <KEY_RING> \
    --location <LOCATION> \
    --primary-version <KEY_VERSION>

The only new flag in this command is <KEY_VERSION> which refers to the version number of the new primary key.

Re-encrypting Data with a New Primary Key

If you have encrypted data with the prior key, that prior key can still be used to decrypt that data. If your key is compromised, your data will be insecure unless you re-encrypt it with your new primary key.

You should do this with the following gCloud CLI command:

gcloud kms encrypt \
    --key <KEY_NAME> \
    --keyring <KEY_RING> \
    --location <LOCATION>  \
    --plaintext-file <FILE_TO_BE_ENCRYPTED> \
    --ciphertext-file <FILE_TO_STORE_ENCRYPTED_DATA>

<FILE_TO_BE_ENCRYPTED> should be the local file path for reading the plaintext data.
<FILE_TO_STORE_ENCRYPTED_DATA> should be the local file path for where you plan to save the encrypted output.
If you want to verify that your encryption is now using the new primary key, you can test it by running the decrypt command.

Disabling or Destroying the Prior Key Version

Disabling or destroying a key both remove the key’s functionality. It’s important to ensure that compromised keys are disabled or destroyed.

The difference between the two outcomes is that destroyed keys are removed permanently (after their scheduled destruction date), which means that if you have anything encrypted that relies on that key to be decrypted, and that key is destroyed, you lose access to that data permanently. If you are certain that you no longer need the key, destroying it is a way to clean up your key ring and prevent a compromised key from somehow being restored.

Using the GCP Console:

In the GCP Console, you can disable and destroy a key by following these steps:

In the key ring view, click the key you recently rotated.
Next to the version of the key you want to change, you’ll see an Actions column with three vertical dots. Click on the dots.
Depending on which action you want to take, you can either select Disable or Destroy.
If you choose Destroy, you will need to type in the key name and click Schedule Destruction to confirm the action. Once you have done this, you will have fully rotated your keys and cleaned up the prior key version.

Using the gCloud CLI:

You can also disable or destroy keys with the CLI

You can use this command to disable a key version:

gcloud kms keys versions disable <KEY_VERSION> \
    --key <KEY_NAME> \
    --keyring <KEY_RING> \
    --location <LOCATION>

And you can use this command to destroy a key version:

gcloud kms keys versions destroy <KEY_VERSION> \
    --key <KEY_NAME> \
    --keyring <KEY_RING> \
    --location <LOCATION>

If you run the destroy a key version command, it will be scheduled for destruction. You can 24 hours after that to change your mind and restore the key.

Using No-Code Steps in Blink to Rotate GCP Keys

If you need to manually rotate access keys, you will need to remember each step and stop what you are working on to ensure you do it all properly. Working through these steps each time isn’t hard, but it takes time.

With Blink, you can easily create an automation that rotates access keys, re-encrypts files that are using the prior key version, and disables the prior key version with a simple click. If a key is compromised, you’ll be able to act quickly.

Blink also allows you schedule disabled keys for destruction after a certain period of time. Ensure that your keys are cleaned up while also giving your team time to validate that you no longer need the old versions.

Create your free Blink account and make it easy to rotate your GCP keys.

How to Find and Remove Unused AWS Passwords

Patrick Londa — Tue, 22 Nov 2022 20:12:14 +0000

If an AWS user has a password to login to the AWS console, but hasn’t used it in over 6 months, their login credentials might be a security liability to the rest of your account.

By running a check to find unused passwords, you can delete login profiles and reduce your account’s potential attack surface.

In this guide, we’ll show you how to find and delete unused passwords to strengthen the security of your AWS account.

Finding and Deleting Unused AWS Passwords

You can find unused passwords with the AWS Console, the AWS CLI, or the AWS API.

Using the AWS Console:

Here are the steps to find and disable unused passwords:

Step 1. To start, log in to your AWS IAM Console.

Step 2. In the navigation pane, select Credential report.

Step 3. When you click Download Report, you’ll get a CSV file using the naming structure status_reports_T.csv.

Step 4. Filter on the fifth column named password_last_used.

If they have N/A, it means they have no password assigned.
If they have no_information, it means they haven’t used their password since IAM started tracking passwords (Oct. 20th, 2014).
If they have a date that is earlier than a threshold you set (e.g. 90 days), then you can consider their passwords unused and act on them.

Step 5. Go to the navigation pane and select Users.

Step 6. Select the name of a user who has an unused password.

Step 7. Go to the Security credentials tab.

Step 8. Under Sign-in credentials, click Manage next to Console password.

Step 9. Select Disable for Console access, then click apply.

Step 10. Repeat steps 5-9 for all users you identified as having an unused password.

Using the AWS CLI:

If you would prefer to do this using the AWS CLI, here are the steps.

Step 1. You can find unused passwords by running the following command:

aws iam list-users

This will output a list of all users in your AWS account. In the output, you’ll see information about all users, including a PasswordLastUsed value. If the user has no value listed, then they either do not have a password or haven’t used their password since tracking began (Oct. 20, 2014).

Alternatively, you can use the get-user command if there is someone specific you suspect might have an unused password.

Here is the output of running the list-users command:

"Users": [
    {
        "UserName": "Charlie",
        "Path": "/department_abc/group_def/",
        "CreateDate": "2017-06-19T10:01:44Z",
        “PasswordLastUsed”: “2022-10-23T11:01:29Z”,
        "UserId": "AID3YDW8DMLG72PEANUTS",
        "Arn": "arn:aws:iam::123456789012:user/department_abc/group_def/Charlie"
    },
    {
        "UserName": "Lucy",
        "Path": "/department_abc/group_ghi/",
        "CreateDate": "2018-03-09T13:21:33Z",
        “PasswordLastUsed”: “2018-05-21T13:21:51Z”,
        "UserId": "AIDIODN4U1W727PEANUTS",
        "Arn": "arn:aws:iam::123456789012:user/department_abc/group_ghi/Lucy"
    }
]

You can now see that Lucy has not used her password to log in to AWS in multiple years.

Step 2. Next, you can run the following command to delete the password for anyone who, like Lucy, has not used their password in a certain amount of time:

aws iam delete-login-profile 
--user-name Lucy

The result of this command is that you have denied the user the ability to sign in to the AWS Console, which limits your security risk if an old password were to become compromised. They will still have access to the AWS CLI and API, so make sure to also remove their access keys.

Using the AWS API:

You can also use the AWS API to find and delete unused passwords.

Step 1. Use the ListUsers action to get a list of all users in your AWS. You can use the PathPrefix parameter to narrow the list of users.

Here’s an example of that request:

https://iam.amazonaws.com/?Action=ListUsers
&Version=2010-05-08
&AUTHPARAMS

Here’s an example response:

<ListUsersResponse xmlns="https://iam.amazonaws.com/doc/2010-05-08/">
 <ListUsersResult>
    <Users>
       <member>
          <UserId>AID3YDW8DMLG72PEANUTS</UserId>
          <Path>/department_abc/group_def/</Path>
          <UserName>Charlie</UserName>
          <Arn>arn:aws:iam::123456789012:user/department_abc/group_def/Charlie</Arn>
          <CreateDate>2017-06-19T10:01:44Z</CreateDate>
          <PasswordLastUsed>2022-10-23T11:01:29Z</PasswordLastUsed>
       </member>
       <member>
          <UserId>AIDIODN4U1W727PEANUTS</UserId>
          <Path>/department_abc/group_ghi/</Path>
          <UserName>Lucy</UserName>
          <Arn>arn:aws:iam::123456789012:user/department_abc/group_ghi/Lucy</Arn>
          <CreateDate>2018-03-09T13:21:33Z</CreateDate>
          <PasswordLastUsed>2018-05-21T13:21:51Z</PasswordLastUsed>
       </member>
    </Users>
    <IsTruncated>false</IsTruncated>
 </ListUsersResult>
 <ResponseMetadata>
    <RequestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestId>
 </ResponseMetadata>
</ListUsersResponse>

Step 2. Now that you can see which users have passwords that are no longer being used, you can delete their login credentials with the action DeleteLoginProfile.

Since in the example above, Lucy hasn’t used her password to login to AWS since 2018, we can go ahead and delete her login profile.

Here’s an example of that request:

https://iam.amazonaws.com/?Action=DeleteLoginProfile
&UserName=Lucy
&Version=2010-05-08
&AUTHPARAMS

Here’s an example response:

<DeleteLoginProfileResponse xmlns="https://iam.amazonaws.com/doc/2010-05-08/">
  <ResponseMetadata>
    <RequestId>7a62c49f-347e-4fc4-9331-6e8eEXAMPLE</RequestId>
  </ResponseMetadata>
</DeleteLoginProfileResponse>

Now, you have successfully prevented unused passwords from being used to log in to the AWS Console if they become compromised.

As we mentioned in the other methods, you will still need to separately look to see if the user has unused access keys. You can do that with the ListAccessKeys action.

Finding Unused Passwords Automatically with Blink

You can find unused passwords manually by following the steps above, but that relies on you taking the time to set reminders and manually update each user. It’s time-intensive and requires context-switching.

With Blink, you can easily create an automation that runs on a schedule to find passwords that have not been used in a certain number of days. You can then kick off a Slack notification that makes deleting their login profile as easy as clicking “approve”.

By automating this entire workflow, you can turn a best practice into a built-in workflow.

Create your free Blink account and boost your AWS security posture today.

Reducing Your Cloud Costs: An Operational Optimization Guide

Patrick Londa — Wed, 16 Nov 2022 16:57:36 +0000

Are your cloud costs on the rise?

In this cost optimization guide, we outline the most common operating expenses and provide you with actionable recommendations to lower your monthly cloud bill.

Cloud costs are top of mind as many business leaders and teams are focusing attention on honing their operational efficiency.

In April at CIO.com’s Future of Cloud Summit, Dave McCarthy, research vice president of cloud infrastructure services at IDC, shared that cloud spending represents roughly 30% of current IT budgets. In the 2022 State of Cloud Report by Flexera, 750 surveyed executives shared that they estimate they are wasting 30% of their cloud spend, while also saying that they expect costs to increase 47% over the next year. If you combine those stats, there is an efficiency opportunity roughly the size of 10% of IT budgets.

Achieving those cost savings isn’t as easy as flipping a switch. There is wasted spend embedded across multiple resource types, regions, and services. By function, the main categories of cloud spending are compute time, data storage, and data transfer.

In this post, we’ll outline a framework for reviewing your cloud spending today, identifying wasted resources, and reviewing your long-term infrastructure efficiency.

Reviewing Your Current Spending

“What are we currently spending money on?”

To start, you can review your current spend at the account-level with the major cloud providers. AWS, Azure, and GCP all have reporting options that enable you to view and filter your spending over a period of time.

In AWS, you can create Cost and Usage Reports. In GCP, you can review your Cloud Billing Report and view spend by “Project” or other filters. In the Azure portal, you can download usage and charges from the “Cost Management + Billing” section.

These views may be useful to get started and see transactional costs, such as from data transfers. In order to get more granular details on your cloud spending, you should leverage resource labels and tags to accurately categorize expenses.

With labels and tags, you can associate resources with specific cost centers, projects, business units, or teams. You can then easily organize your resource data, create custom reports, and run specific queries.

If you do not currently have a mechanism or standard practice around resource tags and labels, you can refer to these how-to guides for setting up mandatory tags:

AWS: Enforcing Mandatory Tags Across Your AWS Resources
GCP: Enforcing Labels and Tags Across Your GCP Resources
Azure: Enforcing Mandatory Tags Across Your Azure Resources

If you use more than one cloud computing provider, you’ll need to aggregate invoices and usage reports across vendors. In this scenario, having consistent tagging methods across platforms is even more useful as it can offer a consistent way to view your resource usage and expenses.

Once you have a clear sense of your current spending, you can look for opportunities to reduce your expenses.

Eliminating Unnecessary Resources

“What resources are we spending money on and not using at all?”

As projects are spun up and shut down, there are often resources that become unattached and left behind. While they are no longer in use, they are still costing your organization money on a recurring basis.

Ideally, you have an automated way to regularly catch and delete these unattached resources. With a no-code platform like Blink, teams can scale up scheduled automations to continuously detect and remove unnecessary resources.

If you don’t have automations already in place, you can manually review resources in the console and remove unused ones in bulk. It can be time-consuming, but you may be able to reduce your operating costs significantly this way in the short-term.

To know what types of resources to review, here are some common examples:

Finding and removing idle resources is a clear way to cut your operating costs, but it also is an important practice for maintaining a strong security posture. If you leave resources like unattached IP addresses, idle NAT Gateways, load balancers with no target, or orphaned Secrets lying around, bad actors could find them and take advantage of the information. In this way, resource management is key to reducing costs and reducing risk.

Optimizing and Updating Resources

“How can we optimize our existing resources?"

Now that you’ve reviewed and removed unused resources, you can now look at optimizing the resources you are using.

Using the Right Family for the Job

Whether you are creating new resources or evaluating existing ones, it’s important to consider which family of resources best fits your needs. If you’re using general-purpose machines, there might be another more cost-effective machine that is a better fit.

Depending on your usage, you may need more capacity in some specifications than others. For example, if you’re using AWS, there are Compute Optimized instances under the C family (e.g. EC2 C7g instances) which offer optimal price performance for especially computing-intense use cases, like batch processing workloads and scientific modeling. Other families include Memory Optimized (e.g. EC2 R6a instances) and Storage Optimized (Ec2 lm4gn instances). There are lots of other families (e.g. IOPs, network, accelerator-optimized) depending on the platform and the specification you want to optimize for.

When considering your performance requirements, you might have use cases like batch jobs or workloads that are fault-tolerant. Azure, GCP, and AWS all have unused capacity that they offer as less expensive, less reliable Spot VMs. Compared to on-demand instances, they are up to 90% less expensive to run.

Updating to New Machines

Within each of these families, there are often newer versions being offered. Often, the newer versions run more efficiently or have higher performance, so it’s a good best practice to upgrade to newer versions as much as you can.

One example of this is with EBS volumes. By switching from EBS GP2 volumes to EBS GP3 volumes, you can reduce your costs by 20%. There are some small performance tradeoffs, but it’s important to keep these types of upgrade opportunities in mind.

Another AWS example is switching from older machines to ones that use the new AWS Graviton2 processors. Instances running on Graviton2 processors vs. Intel processors offer up to 40% better price performance, with specific efficiencies varying by family.

Looking for Low CPU Usage

One way to optimize your spending is by rightsizing resources to match the usage level that you need. For example, you may be running an instance or virtual machine that has more computer capacity than you need.

By reviewing your usage data, you can determine if you are running at an average CPU usage of 30% or less for example. By reducing the size or type of instance, you can slightly reduce your spend, which adds up over time.

Here are some how-to guides that show examples for each platform:

AWS: Finding and Resizing Amazon EC2 Instances with Low CPU Usage
GCP: Finding and Resizing GCP Compute Instances with Low CPU Usage
Azure: Finding and Resizing Azure Virtual Machines with Low CPU Usage

Using Long-Term Resourcing for Predictable CPU Usage

Another way to optimize your costs is by leveraging reserved instances or committed use discounts. In exchange for predictable computing expectations, the major cloud providers offer resources at a discount with a committed term, such as 1 year or 3 years.

Here are some how-to guides that show examples for each platform:

AWS: Lowering Costs on Long Running AWS EC2 Instances
GCP: Lower Costs for Long Running GCP Instances with Committed Use Discounts
Azure: Optimizing Costs for Long Running Azure VMs with Reserved Instances

Starting Nightly Non-Production Scale-Downs

Are there any resources that you can shut-down when they are not being used? For example, if your team is working with a test environment during certain work hours, you don’t need to run it 24 hours a day. You can scale it down at night and scale it back up the next morning.

With some automation, pausing and restarting a non-production cluster could be as simple as clicking an approval button in a slack message, and reducing your daily cloud costs.

Here are a couple examples of how to pause and restart clusters nightly:

AWS: How to Scale Down AWS EKS Clusters Nightly
GCP: How to Pause Your GKE Cluster Nightly
Azure: How to Pause Your AKS Cluster Nightly

Storing and Moving Data Efficiency

“Can we optimize how our data is stored and transferred?”

Storing Only Relevant Data

Your cloud bill is also impacted by how much data you are storing. While it’s useful to collect data to see how your services are running, it likely becomes less useful and relevant over time. Even if you want to maintain as much data as possible, you’ll want to employ a strategy of periodically switching data over to less-costly, long-term storage vehicles, such as Amazon’s S3 Glacier storage.

Here are some how-to guides for AWS on how to identify data that hasn’t changed in a while and how to reduce logging storage costs.

AWS: Detecting AWS DynamoDB Tables with Stale Data
AWS: Lowering AWS CloudTrail Costs by Removing Redundant Trails
AWS: Ensuring AWS CloudWatch Log Groups Have Set Retention Periods

Optimizing Data Transfers

Data transfers may also account for a significant part of your cloud costs, and vary greatly depending on their source, destination, method of transport, and size.

You can also likely expect charges if you are transferring data across regions or across availability zones. Unless your business case requires it, you should look to avoid data transfers that go across regions and availability zones.

While inbound (or ingress) data transfers between the internet and your cloud provider are not charged, outbound transfers are charged per service. You should reduce outbound data transfers from your cloud to external destinations as much as possible.

If you are transferring data across AWS services for example, you should be utilizing private endpoints. This way, when you are accessing a S3 bucket from an EC2 instance, you can avoid data transfer charges.

The same principle applies for transferring data from your cloud to on-premises locations, and tools like AWS Direct Connect, GCP Direct Peering, and Azure ExpressRoute which may offer lower cost per GB compared to transfers over the internet. Actual savings depends on the amount of data you are moving, and if you are below a certain threshold, it might not make sense.

You can read more about the types of data transfer charges in the Cost Optimization pillar of the AWS Well-Architected Framework, or these AWS, GCP, and Azure resources.

Achieving Operational Excellence with Blink Automations

So far, we have covered several areas where you and your team can focus and optimize your costs, but significant savings over time takes new processes.

Beyond finding unused resources, you need an automated process for alerting you to cost reduction opportunities, and then making approval for removing resources as easy as clicking a button. If you only rely on scripts, you may accidentally take down environments or orphaned resources that should have been left up.

With Blink, you can use no-code automations to achieve operational excellence. In the cost optimization context, Blink lets you create and run dozens of common resource checks and send reports to email or Slack channels with simple, actionable options.

By running these Blink automations on a schedule, you’ll be able to confidently ensure that you are achieving operational excellence not just one time, but daily. You can take the same Blink automation approach for other operational excellence categories, like security operations, incident response, troubleshooting, and permissions management.

Get started with a free Blink account or reach out to us directly to hear more.

Switching from gp2 Volumes to gp3 Volumes to Lower AWS EBS Costs

Patrick Londa — Mon, 17 Oct 2022 14:18:16 +0000

In December 2020, Amazon Web Services (AWS) announced the availability of gp3, the latest version of general purpose SSD volumes for Amazon Elastic Block Store (EBS).

Prior to this, there had been only a linear relationship between EBS volume performance and storage capacity, which led to over-provisioning ⁠and non-cost-effective pricing. Now, with gp3, if you require higher IOPs (input/output operations per second) and throughput, you don’t need to also pay for excess storage.

When you switch from gp2 volumes to gp3 volumes, you lower your costs by 20%. For example, in the us-east-1 region, gp2 volumes cost $0.10/GiB-month compared to gp3 volumes at $0.08/GiB-month.

With gp3 volumes, you get a baseline performance of 3,000 IOPs and 125MB/s at any volume size, with the ability to scale up to 16,000 input/output operations per second (IOPS) and 1,000 MiB/s for additional fees. With high performance and lower costs, making this switch is easy to justify.

In this guide, we’ll show you how you can switch your gp2 volumes to gp3 to lower your costs by 20%.

How to Switch from GP2 Volumes to GP3 Volumes

You can make the switch from GP2 volumes to GP3 volumes by either using the console or the AWS CLI, and you won’t disrupt your running EC2 instance.

Using the AWS console

Open EC2 console.
Choose Volumes.
Select which gp2 volume you need to modify.
Choose Actions ---- Modify Volume.
Once your chosen volume's ID and current configuration are shown in Modify Volume window, set your desired configuration values as follows:

a. Modify type by choosing gp3 for Volume Type.

b. Modify size by entering a new value for Size.

c. Modify IOPS by entering a new value for IOPS.

d. Modify throughput by entering a new value for Throughput.

e. Having completed changes to volume settings, choose Modify.

f. Choose Yes, when prompted for confirmation.

You can repeat these steps for as many gp2 volumes as you have to scale this cost savings.

Using the AWS CLI

Use the modify-volume command so you migrate to gp3. This is an example of an 8-GiB gp2 to gp3 migration command:

aws ec2 modify-volume --volume-type gp3 -volume-id vol-11111111111111111

The output should look like this:

{
  "VolumeModification": {
    "VolumeId": "vol-11111111111111111",
    "ModificationState": "modifying",
    "TargetSize": 8,
    "TargetIops": 3000,
    "TargetVolumeType": "gp3",
    "OriginalSize": 8,
    "OriginalIops": 100,
    "OriginalVolumeType": "gp2",
    "Progress": 0,
    "StartTime": "2021-02-03T13:38:08+00:00"
  }
}

By running this command and making this change, you have now reduced your monthly AWS costs for this volume. You will need to repeat this process for each gp2 volume.

Automating Your Migration to GP3 Volumes

Switching your gp2 volumes over to gp3 volumes is a simple way to lower your costs, but the process requires manual action on each volume. If you have dozens or hundreds of gp2 volumes, this can be highly time-consuming. And then what happens if an even more cost-effective gp4 volume is announced?

For migrations like this, automation can save you significant time and make these optimizations more feasible at scale.

When you create a free Blink account, you can run automations that identify and alert you to any gp2 volumes associated with your EC2 instances. You can then click a button in Slack and migrate that volume to gp3 instead.

Get started with Blink and make it easy to optimize your cloud costs.

The Ops Community ⚙️: Patrick Londa

Aligning Your AWS Account with the FFIEC Cybersecurity Standards

Understanding the FFIEC Cybersecurity Standards

FFIEC Cybersecurity Guidance for AWS

Automating FFIEC Compliance for AWS with Blink

Introducing Blink Copilot: Generative AI for Security Workflows

The Critical Need for Security Automation

Automate Security Beyond the SOC

Powerful No-Code Security Automation

Get Started with Blink Copilot

How to Find and Update Public EC2 AMIs in AWS

Understanding AMIs for AWS EC2

How To Find Any Public AMIs in AWS

Using the AWS Console:

Using the AWS CLI:

Changing Public AMIs to Private AMIs in AWS

Using the AWS Console:

Using the AWS CLI:

Automating AWS Permission Checks for EC2 AMIs with Blink

How to Search for an IOC Across Devices in CrowdStrike

Searching for an IOC Across CrowdStrike Hosts

Using the CrowdStrike Console:

Using the CrowdStrike Falcon API:

Automatically Searching Across Devices for IOCs with Blink:

How to Get User Activity From Your Azure Logs

How to Get User Activity From Azure Logs

Using the Azure Monitor Log:

Using the Azure CLI:

Getting User Activity from Azure Faster with Blink

How to Verify IOCs and Enrich Your Incident Response with VirusTotal

Verifying IOCs Using VirusTotal

Using VirusTotal on a Browser:

Using the VirusTotal API:

Taking Action Based on VirusTotal Reports with Blink

How to Migrate from AWS EC2 Launch Configurations to Launch Templates

Launch Configurations vs. Launch Templates

Launch Configurations

Launch Templates

How to Migrate Launch Configurations to Launch Templates

Migrating One Launch Configuration

Migrating All Launch Configurations‍

Updating Auto Scaling Groups to Use Launch Templates

Using the AWS Console:

Using the AWS CLI:

Replacing Launch Configurations Automatically with Blink

How to Enable Autoscaling for a GKE Cluster

Understanding Your GKE Autoscaling Options

Minimum and Maximum Nodes

Balanced vs. Optimize-Utilization

Configuring Autoscaling for an Existing Node Pool

Using the GCP Console:

Using the gCloud CLI:

Enabling Autoscaling When Creating a New GKE Cluster or Node Pool

Using the GCP Console:

Using the gCloud CLI:

Checking Whether Autoscaling is Enabled with Blink

How to Manually Rotate Keys in GCP

Manually Rotating Keys in GCP

Using the GCP Console:

Using the gCloud CLI:

Re-encrypting Data with a New Primary Key

Disabling or Destroying the Prior Key Version

Using the GCP Console:

Using the gCloud CLI:

Using No-Code Steps in Blink to Rotate GCP Keys

How to Find and Remove Unused AWS Passwords

Finding and Deleting Unused AWS Passwords

Using the AWS Console:

Using the AWS CLI:

Using the AWS API:

Finding Unused Passwords Automatically with Blink

Reducing Your Cloud Costs: An Operational Optimization Guide

Reviewing Your Current Spending

Eliminating Unnecessary Resources

Unattached Disks

Unattached IP Addresses

Old Snapshots

Optimizing and Updating Resources

Using the Right Family for the Job

Updating to New Machines