Simplifying DynamoDB Backup and Restore

Introduction

Amazon DynamoDB is one of the fastest growing and most widely adopted databases in the AWS cloud. Due to its support for very high transaction volumes (20 million requests per second), it has become an important function of mission critical workloads in the cloud. Several large enterprise customers like Lyft, Airbnb, Redfin and we (Clumio) use it extensively to build production applications and deliver an ideal experience to customers. Since this database has critical persistent data, its protection has become necessary for all customers to ensure that they can recover from an outage or data loss incident.

Since Amazon DynamoDB is offered as a service, there are no manual scripts that customers can use to protect the database. To solve this issue, DynamoDB itself provides multiple options to ensure that customers can backup their database. DynamoDB provides Point in Time Recovery (PITR) configuration for all of its tables. When configured, customers can recover to any specific point in time in the last 35 days, depending on the PITR configuration. Additionally, DynamoDB allows customers to take a manual snapshot of the entire database so that customers can recover from that snapshot at any time in the future. This snapshot can also be used to achieve compliance with certain regulatory requirements that are enforced upon some industry verticals. However, while speaking to enterprise customers, we’ve heard them ask for following enhancements:

• Simpler Ransomware Protection with Air Gap: In-account snapshots provide limited protection, and vaulting of snapshots to avoid accidental deletion can be prone to human error at scale.
• Built-in Long-Term Compliance Backup: Automated backups are limited to 35 days and manual scripting is required to store data long-term.
• Enhanced Visibility: Customers are looking for a centralized place to get complete visibility of which DynamoDB Tables are backed up successfully, when they were backed up, what DynamoDB Tables are compliant, etc.
• Full snapshots: Providing a backup solution for long-term retention requires scripting and manual snapshots to be stored in S3. Every time an export or a snapshot is taken, it’s a full copy of the data, increasing costs exponentially.
• Faster Compliance Validation: Customers require tools to quickly show that DynamoDB Tables are compliant during audit events. Auditors typically want a compliance report but also require the audited organization to easily prove compliance by pulling some/all information from their Tables.
• Logical table recovery: DynamoDB typically consists of multiple logical tables inside the Table, and recovery of the entire Table increases RTO and cost significantly.

All of the customer conversations converged to demonstrate that customer’s require enhancements for key capabilities like ransomware protection, compliance visibility, granular recovery with low RTO, and built-in efficiencies to drive down costs. To help with these customer needs, we are excited to announce the newest innovation in the Clumio family with Clumio Protect for DynamoDB.

Clumio Protect for DynamoDB provides the following benefits:

• Continuous DynamoDB Backups with Air Gap Ransomware Protection: Manage operational recovery in-account with Clumio Snap and extend with Clumio SecureVault for turnkey air-gapped protection against ransomware and other account compromises.
• Lower RTO and RPO with Flexible Recovery: With multiple point-in-time backups inside SecureVault, you can lower your RPO allowing for finer grain recovery, and reduce RTO allowing you to get back up and running quickly to any AWS account within Clumio.
• Incremental Long-Term Retention with Granular Recovery: Use direct query to access DynamoDB tables stored for long-term retention, removing the need to restore and reconstruct old S3 data. Clumio backups are incremental and provide simple access for compliance, eDiscovery, or legal matters pertaining to DynamoDB data.
• Global Visibility and Risk Profile Analysis: Global visibility of all DynamoDB tables across all your accounts, with a ransomware risk profile and recovery score showcasing any assets unprotected or vulnerable to ransomware.
• Compliance Reporting: Simplify audits using easy reporting with simple and intuitive recovery of long-term retained data.

Similar to data protection of other AWS assets, the deployment of the solution is simple with a single pane of glass experience. The deployment is infrastructure-less and agent-less for all your AWS accounts via a Cloud Formation Template or Terraform. After deploying, you can see all your DynamoDB Tables across all your accounts. Next, you create a Policy with your required RPO configurations and easily apply that to one or all of your AWS accounts with the configuration of a simple Rule.

Global Policy and Protection Rules

Policy allows you to define your business intent of configuring when to create backups and how long to store them in Clumio. You even get the option to configure On-Demand backup options and whether to keep the backups In region or a Specific Region of your choice. The Snap tab allows you to configure Operational Recovery operations like Point-In-Time recovery (PITR) and In-account snapshots.

With Protection Rules, you can easily extend the policy to any/every DynamoDB Table of your choice. You can protect your DynamoDB Tables either by Account, Region or Tags. With Tags, you also get the flexibility to configure specific tags, multiple tags, All of the tags or Tag Contains. This rule provides complete protection of Tables that are present not just today but also any future tables with appropriate tags.

Data Recovery

While performing Data recovery, you can either recover from Clumio’s SecureVault Backups or from Clumio Snaps. If you have the PITR configured, you can also recover that or any previously taken snapshots for the DynamoDB Table, all from the same calendar view. With the intuitive interface, you get complete visibility into when  backups were taken and whether any  backup is missing from the view. While performing recovery from the SecureVault backups, you can either restore the entire DynamoDB Table or perform Granular recovery.

When performing Full recovery, you get the following choices:

• Configuring a Table name
• Selecting a specific Account/Region (recover into any account in any region)
• Specifying Tags
• Selecting the Encryption Type
• Restoring Local and Global Secondary Indexes

While performing Granular recovery, you get the following experience:

You can specify a particular Partition Key along with an optional Sort Key. Add further filters in Advanced Options to be able to quickly preview the data that matches your specified filters. Results are automatically hidden to ensure data privacy, and the administrator needs to have access to a particular role in order to perform  Granular Recovery.

In the next step, you can either directly download the data or even transparently share the data with another colleague inside the organization, or an auditor, to ensure compliance.