Maximize Amazon EBS and EC2 Performance with Clumio Discover

Amazon EBS is a popular cloud-based block storage system, generally used in conjunction with Amazon EC2. Amazon EBS volumes can be used for web applications, databases, and file systems to name a few. Amazon EBS volumes vary by volume size, disk I/O, throughput and latency, and are billed per GB per month. As an Amazon EBS customer, you are billed for the total storage provisioned to your account, even if it isn’t used. For example, if you provision a 100-GB Amazon EBS volume for a month, but store only 10 GB of data on that volume, you are still billed for 100 GB-month. The 100GB storage is called the provisioned size and the actual storage of 10GB on the volume is called the utilized size.

Why Utilization Matters

Utilization metrics help identify volumes and instances that need to be changed, upgraded, or even downsized so that they meet your requirements in an efficient and cost-effective manner. Furthermore, insights on utilization at total and asset-levels enable users to understand their backup needs and spend. (Asset refers to an individual volume or instance.)  

We’re thrilled to announce that Clumio, known for its immutable and air gapped data protection for AWS, now offers advanced utilization metrics for Amazon EBS and EC2 that will help users better utilize their AWS infrastructure. 

The new feature is available as part of Clumio Discover, and helps customers with the following insights:

• Total storage breakdown – View and analyze total provisioned and utilized storage on Amazon EBS volumes and instances by type across all AWS accounts to understand if you are taking full advantage of the array of storage options AWS offers.

• Asset-level provisioned size – Get insights on each asset’s provisioned size in AWS to understand the storage provisioned.

• Asset-level utilized size – Get insights on each asset’s utilized size to understand the actual AWS storage of the asset.

• Summarized storage– High-level insights on total provisioned and utilized sizes of all volumes and instances.

• Global filtering – Drill down on specific volumes or instances across AWS accounts and regions.

• History of storage – View a history of provisioning and utilization of volumes and instances over a specified period of time.

Let us walk through the different interfaces in the product that provide you with the aforementioned benefits.

Total Provisioned and Utilized Sizes

You can get a quick preview of your total provisioned and utilized sizes of volumes and instances on the Inventory Summary page. Additionally, apply filters for specific accounts, regions and tags to synthesize custom insights.

Total Storage Breakdown

Given the number of options AWS offers for choosing a volume or an instance type, oftentimes users end up paying higher prices by using the wrong volume or instance type for their needs. We solve this pain point by providing a simple view of your total storage by different types of provisioned volumes and instances.

History of Provisioning and Utilization

Get easy access to your storage history by viewing metrics for the last 7 days, 30 days, or 90 days. This is helpful in making future infrastructure-related decisions by predicting the provisioning and utilization of your storage needs.

Individual Asset Data

A unified view across all your volumes and instances is beneficial in auditing to see utilization of your provisioned storage. This will help drive important optimization decisions by migrating to smaller volumes.

The next section of this blog explores how Clumio calculates all of the underlying measurements that power this data.  This information will be helpful as I will refer to it in upcoming blogs expanding on the concepts of utilization, change rates and growth rates.

Amazon EBS Snapshot-based Methodology

Amazon EBS snapshots are a point-in-time copy of data on a given volume, and each snapshot contains all of the information needed to restore that data to a new volume. Clumio uses this to infer the utilized size of any given volume based on its recent snapshot.

Let’s assume that you have provisioned an Amazon EBS volume for 10 blocks on Jan 01, 2022. An application write to block 0 through block 4 on Jan 02, 2022 results in 5 blocks being utilized. A snapshot taken after these volume writes will hold the 5 blocks of data on the volume. This is referred to as the snapshot’s utilized size. EBS snapshots are incremental in nature, which means that only the blocks on the volume that have changed after the most recent snapshot are stored. Therefore, subsequent snapshots store the changed blocks (delta) and AWS charges customers for these delta sizes of the snapshots.

 

Provisioned volume blocks on Jan 01, 2022	Utilized volume blocks on Jan 02, 2022
Block 0	A0
Block 1	B0
Block 2	C0
Block 3	D0
Block 4	E0
Block 5
Block 6
Block 7
Block 8
Block 9

 

How Clumio Discover Calculates the Utilized Size of an Amazon EBS Volume:

Consider the volume represented below with a number of snapshots taken in different months. The sizes represented are utilized sizes of snapshots which are indicative of the volume size if you were to create a new volume from any snapshot. As mentioned above, EBS snapshots store the delta sizes and Clumio calculates the utilized sizes of the snapshots by making ListSnapshotBlocks API calls.

The most recent snapshot taken on Aug 01, 2022, is the snapshot that we use for the volume’s utilized size as this snapshot indicates its latest state. Therefore, the volume’s utilized size is based on snapshot S6: 30GiB utilized of 100GiB provisioned.

How Clumio Discover Calculates the Utilized Size of an Amazon EC2 Instance:

Consider the instance represented below with two attached volumes that have a number of snapshots taken in different months. The instance’s provisioned size of 200GiB is derived from the attached volumes’ provisioned sizes, and snapshot sizes are utilized sizes of snapshots. 

For calculating the utilized size of the instance, we find the most recent snapshot of each attached volume and add them up. Therefore, the utilized size of the instance is based on snapshot S6 (from attached volume 1) and S3 (from attached volume 2): 32GiB + 50GiB = 82GiB utilized of 200GiB provisioned.

For calculating the total provisioned and utilized sizes across volumes and instances, we simply add up the individual sizes of volumes and instances.

Give Clumio Discover a try today to improve visibility and understand your AWS provisioning and utilization strategies for volumes and instances. Stay tuned for additional blog posts in the future where I will delve into topics like volume and instance change rates and growth rates.