Your Cloud Backup Journey with AWS Backup & Snapshots

After countless conversations with customers about backups, there is a similar journey that they have all taken from using AWS snapshots to third-party snapshot managers to custom scripts, and then ultimately to their final destination: Clumio Protect. This journey started with many bumps and bruises but our customers learned a heck of a lot along the way. This post will share with you the steps most have taken as they stepped into cloud-based backup. Over the following weeks, I will dive into more technical details that will help you learn from those that have traveled the road before you.

You are in the Cloud!

You likely began in the cloud by “Shadow IT” projects or by a “Digital Transformation” initiative that had cloud migration as part of that strategy. Your engineers eagerly spun up some EC2 instances and started putting some database workloads in RDS. Now that your engineering team is well underway, you realize that there is no sound data protection or backup plan for all these new cloud workloads. After all, backup is not typically in an engineer’s vernacular and they assume it’s all taken care of. Everything in AWS is new to you and you have no idea where to start so Google searches dominate your workday.

AWS Snapshots

After some rather quick research, you land on using AWS native snapshots. It seems reasonably priced at $0.05/GB per month and from Amazon, so it should be good enough. You notice you have a nice and painless way to take snapshots using the AWS Management Console. In fact, to make sure your engineers don’t get burned, you jump the gun and take some snapshots on all EBS and RDS volumes with a few clicks on the mouse. Nice. You are starting to think the cloud is pretty awesome.

Your AWS Bill is getting big

A few months go by and you are starting to realize there is a trend that is concerning. Your AWS bill has steadily increased and doesn’t seem to be showing signs of leveling out. You start examining your bill and begin to realize you have little to no idea how it’s going to help you understand what’s driving your AWS backup costs.

After some Google searching and poking around the AWS Management Console, you find that AWS Cost Explorer and CloudWatch may help. You turn those on and wait for the collected data to help you solve your problem. However, after patiently waiting a month you end up being disappointed that all you got was a bunch of graphs and datasets that don’t help you gain the insights needed to understand your costs.

Third-Party Snapshot Managers Hidden Costs

Turning again to searching the web for a better solution, you come up with a couple third-party snapshot managers that show promise. They talk about helping reduce your costs significantly by being able to tier to S3. You see it’s going to help you manage your AWS snapshot policies and tiering older backups to S3. Sounds amazing.

Their cost seems reasonable at $5 per protected EC2 instance per month. To help understand the implications of this license fee for these solutions, let’s look at an example. If you have an average of a 200GB EBS volume per EC2 instance that has a daily backup with a 3% daily change rate, it adds $0.013/GB/month (See Figure 1) to your first month’s EBS Snapshot costs. That’s over a 25% premium over native AWS Snapshot costs. Ouch. And if your EBS volume is smaller, this premium effectively goes up. On top of that they also run an in-account EC2 backup manager instance and temporary worker EC2 instances that add additional usage to your AWS infrastructure and are blended into your bill; Good luck sorting thru that. Sneaky pricing model, huh?

Figure 1: Monthly Snapshot Manager License Fee converted to $/GB per month

S3 Tiering pricier than you think

But you’ll save money on the tiering-to-S3 feature, right? Not so fast. You see, in order to offer this, these third party snapshot managers spin up temporary EC2 instances that run for longer periods of time and add a hidden cost to your bill (in the cloud, time = $$$). They are hidden because it will blend in with your own EC2 bill. The temporary EC2 instances have to traverse thru your EBS Snapshots and copy chunks over to S3.

The break-even on just moving it to S3 tends to be around the three month mark due to all these hidden costs. This means if you have retention periods less than three months like your daily backups, you could actually be spending more than if you had just left it in EBS snapshots. Add in the licensing costs we talked about earlier and it’s starting to feel like you fell for some marketing gimmick.

Because of this break-even point on moving snapshots to S3, if you try to have your daily backups in EBS snapshots and the monthly/yearly backups in S3, this will result in multiple copies of data in both snapshots and EBS (See Figure 2), resulting in a reduction in costs savings from S3. Because EBS snapshots are incremental, it’s hard for the S3 tiering to effectively only move what’s needed to S3. This is due to the fact that daily snapshots (that you didn’t want to move to S3) may point to older blocks that are part of a yearly backup that were already moved to S3. Due to this complexity, it is likely your backup manager has a lot of your data in both EBS snapshots and S3. Do you still think you are saving?

Figure 2: Overlap in costs when using both snapshots and S3-tiering backup scheme

After 6 to 12 months of messing around with AWS snapshot managers, you realize your costs aren’t any better, it’s gotten more complex, and you still don’t really understand what’s driving your backup costs, due to the lack of useful reporting.

Your AWS footprint expands

Meanwhile, your AWS footprint expands as your organization grows more into the cloud. The snapshot manager licensing costs are becoming an eyesore on the bill, and you still don’t have the visibility into what’s driving your costs and where you could potentially save with better backup strategies. You are not quite sure how to address this with the snapshot manager you are using and all the traditional on-prem enterprise backup solutions have haphazard solutions they slapped into the cloud with bloated pricing schemes. Not only that, all the funny math provided to you at the time you deployed your snapshot manager promising savings is evaporating quickly. Seeing the trajectory of where your backup strategy is headed, you look for options.

Custom AWS Backup Scripts

You show your management just how much you are now spending on AWS backups and convince them to give you a shared engineering resource to help develop custom backup scripts. You eventually get what you need up and running and are license free and are finally feeling you have a handle on your backup policies. But as AWS changes their APIs and your business needs adjust (like needing to backup cross-region or protect your backups from ransomware), so do your scripts. Your shared engineering resource becomes a full time role. And now you have a hidden cost of headcount… for backup. Ouch.

Your reporting is still not where it needs to be and you can’t answer simple questions like:

• How much is this snapshot costing me?
• Do I need this orphaned snapshot and how much is it costing?
• Can I project what my backup policies will cost me next month? Next 6 months?
• If I delete this snapshot, how much will my bill go down by?
• At what rate is my data and backups growing?
• What percentage of my EBS volume provisioned capacity is being utilized and needs to be backed up?
• Am I even getting close to meeting my backup compliance requirement?

And then you think, “how exactly did I get here?”

Clumio Protect offers license-free backups

Clumio realized that this customer journey is not ideal and needed to be fixed. This is exactly why we created the first enterprise-grade backup SaaS solution on top of AWS native services that will help customers manage their AWS snapshots for free. On top of that, we help our customers understand what’s driving their backup costs. And for customers that want to easily protect their backups in a secure air-gapped vault, customers can easily move their snapshots into Clumio Protect at a price cheaper ($0.045/GB) than AWS native snapshots ($0.05/GB). It’s no wonder that customers are quickly moving their AWS backups into Clumio Protect.

Figure 3: Clumio Protect offers more backup protection while saving customers on their AWS bill

By moving to Clumio, you get all the benefits you get by moving to a custom solution without the ongoing cost of developing and maintaining it. No hidden license fees, easier administration, better $/GB pricing than AWS native snapshots, and better reporting.

Where are you at in this journey? Why not skip to the ending by moving to Clumio, and move onto bigger and better problems. Start your 30-day free trial. Can’t wait? Contact us and we would be more than happy to help you discover your true backup costs and what they could be.