Amazon RDS Operational Recovery Deep Dive with Clumio

Clumio recently launched backup as a service for Amazon Relational Database Service(RDS). In this three-part blog series, we will dive deep into what this service provides through functional demos of the various protection mechanisms across different RDS databases. We will cover the following aspects as mentioned below:

• Operational Recovery: This is the first order of protection where we use AWS snapshots to backup and recover Amazon RDS databases within an AWS account. The AWS automatic backups are taken every day and it also takes a backup of the logs every 5 minutes for the point in time recovery. This enables full recovery or point in time recovery with an RPO of 5 minutes.
• Extended Retention and Granular Record Retrieval: Organizations need to retain data for the long term to satisfy regulatory requirements, audits, company policies, etc. This Clumio feature lets you do granular record retrieval without the need to recover the complete database. This service will be explored in detail in part 2 of this blog series.
• Rolling Backup: This is a time-lagged standby copy of your RDS database. Here the snapshot is moved to the Clumio service. The rolling backup is an add-on feature that will enable you to do a full restore from backup if the production database is hacked, or the AWS account is compromised. We will explore this in detail in part 3 of this blog series.

In part one of this blog series, we will discuss the operational recovery capabilities with Clumio. Operational recovery leverages AWS snapshots, and we can manage from applying policies to performing recoveries right from Clumio. Many of our competitors call their operational recovery a snapshot manager, and they charge for it, whereas this functionality is free with Clumio.

What does this free tier mean? Imagine this; if you have 100s of databases spread across multiple accounts. Using Clumio Backup as a service, you can manage the protection of Amazon RDS databases using AWS snapshots for free. As a database administrator, you can set up AWS snapshots policies and perform recoveries of the Amazon RDS databases from the Clumio platform. Otherwise, it becomes overwhelming to manage protection so many RDS databases. Isn’t this amazing!!!. The whole AWS snapshot management orchestration is completely provided for free. Let us look into the demo to see how easy and intuitive it is to manage Amazon RDS databases from the Clumio platform.

Please read this interesting blog for more information regarding snapshot managers:
Snapshot Managers Exposed; Announcing Clumio Backup as a Service for AWS RDS.

Figure 1 shows the overall operational recovery demo flow. First, we will show how to set up a unified policy in Clumio platform to configure the retention of AWS snapshots. In the next part, we will be doing a point in time recovery. I have used the PostgreSQL RDS database to explain the operational recovery feature.

We use PostgreSQL because it is free and the most popular open-source RDBMS. The PostgreSQL database version used in the demo is 12.2, and I have populated the database using native PostgreSQL load generator pgbench with 500 GB of data. The point in time recovery is shown using a reference table that has timestamp records. This reference table will help us understand the point in time recovery process.

To start, you will need to connect your AWS account to Clumio, which does it by a simple deployment of CloudFormation Template.

Here is the full flow of the demo, which includes two parts.

Setting a Unified Policy for AWS Snapshots

Step 1: The very first step is that we need to connect Clumio to the AWS account. The AWS account connection is done by deploying the Clumio service CloudFormation Template which will enable Clumio services in your account(see Figure 2). It will take us to AWS CloudFormation Stacks console where you can deploy the CloudFormation stack for Clumio with a single click.

Step 2: In AWS, the RDS PostgreSQL, we need to create tags since Clumio uses tags to set up and enforce the policies. For example, here we created the tag “backup” and value as “PITR”** for this PostgreSQL database(see Figure 3).

**We highly recommend creating generic tags like Clumio: Gold and not such narrow scope tags like PITR. Our policies are very potent, and one unified policy can work across multiple data sources types like VMWare, RDS, EBS, Mailboxes. Even with RDS, they can do Operational Recovery(point in time recovery), Rolling backups and Granular Record Retrieval. Creating such narrow scope tags limits our unified policy’s true potential.

Step 3: Now let us create a Unified Policy in Clumio. A policy defines when data should be backed up and how long the backup data should be retained. Clumio’s unified policy allows you to define a single backup policy across multiple data source types. The source types include Amazon Elastic Block Store (EBS) volumes, Amazon Relational Database Service (RDS) resources, Microsoft 365 Exchange Online mailboxes, VMware vSphere virtual machines. Then we set up the policy for RDS databases. We need to provide the AWS snapshot information like the retention of daily snapshots in terms of the number of days and also when this policy should be enforced(see Figure 4). We then create the policy in Clumio.

Step 4: We subsequently enforce this policy through tags. This way, once you tag all of your databases, we can apply the policy across all these databases using tags. Using tags makes it easy to apply and enforce policies at scale. Let us select the tag for which we want this policy to be applied(see Figure 5). Once the policy is applied, we can check the RDS database in AWS and check the new retention days for the automatic backup. The retention days should be changed to what we have selected in the policy.

Step 5: Now, let us connect to the PostgreSQL database and look at the reference table with ten records with timestamps. The timestamps stored in the table are in UTC format. Let us now do a point in time recovery for the first six records. Figure 6 shows the reference table.

Point in Time Recovery

Step 1: With Clumio, we can go to the RDS database section and select the database we want to perform the point in time recovery. It takes us to the recovery window to select the date for the point in time recovery.

Step 2: Select the orange dots in the calendar, which show when the AWS snapshots were taken, and press recover(see Figure 7).

Step 3: Pick a time, and here the time is in local server time, so we need to make sure that this local server time is converted to UTC so we can perform point in time recovery correctly. Here in the demo, since we would like to recover only the top six records, I will pick the time according to it (see Figure 8).

Step 4: During recovery, the database instance name is automatically prefixed with Restored*. We can change it to the name we want (see Figure 8).

Step 5: Once the database is recovered, let us connect to the database and check the reference, and it will have only six records as seen in Figure 9.

Watch the Demo

Summary:

As seen in the demo, it is effortless to set up Clumio Backup as a Service and use it to protect Amazon RDS databases. Clumio gives operational recovery free of cost, so we would highly recommend you try it out. Clumio will help you manage and protect all of your RDS databases across multiple accounts from a single pane of glass with just a few clicks.