Scaling with Clumio: Unleashing Microservices

Here at Clumio, we’re attacking a common enterprise problem in an uncommon way. The problem entails how to protect various on-premises, public cloud, and SaaS workloads while minimizing risk, complexity and cost. In today’s world of managing the myriad apps and data built in different places on various platforms, organizations find that traditional data protection approaches often end up creating more technical debt than the actual intended benefits. In discussing existing backup solutions with customers over the years, I’ve repeatedly heard comments like, “It’s a pain to manage, but it’s the devil we know,” or “We spend way too much time babysitting backups”, or sadly, “We don’t know if our DR plan even works,” and one that came up recently, “We’ve deployed 3 different backup platforms over the past 4 years.” Sound like anything you’ve heard before? More often than not, the crux of the issue is not people or process, but the inability of technology to scale efficiently¹ for variable data protection demands.

Just over 2 years ago, Clumio started with a clean slate to address the data protection scaling challenge head on. By having no roots in the data center or a business model beholden to large capital outlays and tech refresh cycles, we could start fresh and harness the nearly infinite resources of public cloud and its dynamic scaling attributes. We also understood that most IT organizations’ core strengths aren’t in building and running backup infrastructure, but in other value creation opportunities. This modern perspective drove us to build a SaaS platform for enterprise data protection, with zero dependencies on legacy data center components like fixed, stateful appliances and distributed software deployments. If we could take something really complex and pervasive, and tackle it with ingenuity on a cloud scale, then we knew we’d have something special. As a result, we’ve enabled a drastically simplified, scale on-demand backup experience for our customers. Clumio’s data platform is built with cloud-native, decoupled resources that dynamically scale as demands change, with ongoing iterative optimizations, ensuring that neither we nor our customers accumulate technical debt, even under hyper-growth scenarios and as new functionality is rolled out. Since we steer clear of an appliance-based approach that’s bound to fixed resources (physical or virtual, scale up or scale out—they’re still fixed!), it’s easier for us to unlock supplemental services and capabilities without forcing customers to deploy more instances or incur additional costs of infrastructure, whether on-premises or in the cloud. For the occasional work that requires a predefined instance of compute, we containerize apps to optimize resources, portability, and extensibility.

Below are a few specifics on how Clumio’s scaling model translates to a great customer experience:

• Our data processing pipeline is built to maximize the use of serverless compute. Internally, we affectionately refer to this architectural approach as “Woon’s Army of Lambdas.” The pipeline itself is stateless, which allows nearly limitless dynamic scaling at a moment’s notice. This is a fundamental departure from legacy backup providers whose data pipelines are throttled by fixed resources (i.e. CPU, memory, network interfaces, localized databases).
• The Clumio authentication service, policy engine, backup catalog, fingerprint database, backup indices, and backup data repository are built on cloud-native services that reside outside of the customer’s digital estate. These services scale independently of each other, provide immutability, and often leverage serverless functions. For more detailed goodness on this topic, check out Chadd’s outstanding blog.
• During backup, our stateless Clumio Cloud Connectors run efficient, containerized microservices to inventory data sources, communicate with our backend orchestration engine, and read, reduce, encrypt, and transfer data in parallel streams. Each Cloud Connector deploys in about 5 minutes, load balances automatically as more are added, and is a self-managed component of the Clumio service.
• Indexing and verification of customer backups is performed by the Clumio backend separately from the backup data transfer, using auto-scaling resources outside of the customer’s infrastructure and providing another layer of immutability.
• During a recovery operation, we automatically scale serverless requests in parallel to increase restore throughput, reducing time to restore (see scale-out rehydration here). A Cloud Connector automatically scales resources to optimize this process even further.

So how do all of these thoughtful design considerations solve the issue of technology scaling to protect various workloads across multiple clouds and SaaS? Let’s return to the core problems that customers want to minimize:

• Risk: We’ve taken a security-first approach to building our platform. From inherent immutability, end-to-end encryption, and numerous security certifications that we’ve earned in a fraction of the time compared to our competitors, we’ve mitigated many risks of scaling. Since our software development pipeline isn’t bound to an appliance or virtual instances running in a fixed capacity, this enables a rapid pace of innovation leveraging the power of cloud services that are available on demand. As a result, our customers can enjoy a data platform that can adapt quickly to new demands, rather than lagging behind due to legacy constraints.
• Complexity: We’re SaaS, consumed as-a-service. Clumio is so simple that our customers can begin protecting their data in under 15 minutes, from the comfort of their own web browser. Last week, I had a customer who was only protecting VMware servers with Clumio, and then they added protection for AWS workloads with zero questions or intervention from us. Scaling with simplicity!
• Cost: Our engineering mindset has cloud scaling in our DNA. When we want to turn on more services, we simply tap into the nearly infinite wealth of resources in the public cloud. By offering a born-in-cloud SaaS platform that’s built on an inherent pay-per-use, on-demand consumption model, we can protect our customers’ budgets as they scale by way of a simple, predictable subscription—with no separate hardware or software to deploy, ever.

Clumio exists to provide a modern, flexible, and scalable enterprise data platform so customers can continue their cloud journey without introducing risk, complexity and cost—any of which can hinder velocity and innovation. No one wants to accumulate more technical debt, and Clumio is a data protection solution that is, simply put, debt free. Come check us out and challenge our experts to solve your data protection problems in this brave new world. We look forward to connecting with you!

¹What do I mean by ‘scale efficiently’? At a high level, technology offers different scaling types (software, hardware) and approaches (vertical/scale-up, horizontal/scale-out), and one size doesn’t fit all. As a result of software development, tech modernization and adding new lines of business, an organization’s infrastructure and app scaling requirements change. It’s critical for businesses to increase agility in order to respond to the rapidly changing needs of their customers and market dynamics. Today, leveraging public cloud and SaaS are the most effective ways for organizations to unlock the agility of IT services that they need to compete effectively in this era of rapid change. Why? Because public cloud and SaaS offer the most immediate and flexible scaling capabilities, all on-demand and with the least upfront investment.