cloud database innovation

Cloud Databases and The Innovator’s Dilemma

Dana MorCloud Express

On this page:

Data management in the cloud promises to be the most significant innovation in the database field in decades. IT leaders across all verticals are currently evaluating the various cloud database offerings but find it difficult to assess this new technology with conventional methods at their disposal.

Cloud database technology is rapidly advancing. Even though cloud databases are wildly successful in the very limited market of greenfield application scenarios, the much larger and more lucrative traditional database market, however, remains on-premises as enterprise decision makers remain hesitant and are looking for clues that cloud databases are truly the disruptive technology that vendors are advertising.

This article reviews market dynamics and presents the argument that cloud databases exhibit the very specific characteristic of a truly disruptive innovation, face the same obstacles as other disrupters, and predict their ability to take substantial market share from the incumbents over the long run.

This article discusses the following points:
  • The concept of Disruptive Innovation
  • Why some disruptors succeed and others fail
  • Cloud databases are the first real innovation in the database field in decades

The Concept of Disruptive Innovation

High-tech experts and investors have used The Innovator’s Dilemma, authored by Harvard economist Clayton Christensen some 20 years ago, for identifying truly disruptive technology. The framework explains how established vendors get disrupted by upstarts who often innovate on a shoestring budget in ways that seem out of reach for established incumbents. And, once a technology reaches a tipping point, it disrupts the established market and its products—not gradually—but radically and at rapid pace.

The framework identifies two vital ingredients as the key markers of a successful disruptive innovation:

  • Disruption on price: The dominant, incumbent technology is expensive and requires a highly specialized skill set to operate, and is, therefore, only accessible to a limited upmarket clientele.
  • Disruption along a new dimension: The innovator does not improve on the incumbent product but innovates along a dimension initially not viewed as critical, or even seemingly detrimental to success.

The second point above may seem arbitrary at best—or, counterintuitive at worst—but it is a consequence of basic market dynamics. Building a superior product and, at the same time, charging only a fraction for it will destroy most companies. In fact, several public companies have sought shelter in going private when adjusting from a perpetual licensing model to subscription pricing in order to escape the wrath of Wall Street while restructuring their revenue model.

Cloud databases offer the latest in hardware advances, software upgrades and bug fixes, and fully managed services including maintenance operations. Customers need not bother about backups and other much loathed activities.

Two key technology examples for the genesis of successful disruption and for the market forces it can unleash are the development of microprocessors and miniaturization of storage media. Each one was initially considered an inferior product that could not rival the incumbent; however, within a short period of time, the disrupters were able to accelerate product development and create an entirely new market, eventually toppling the incumbents.

Why Some Disruptors Succeed – and Others fail

There are no shortcuts in this process as exemplified by cautionary tales of failed attempted disruptions, such as HP’s Kittyhawk, a hard disk drive specifically engineered for disruption. Its creators tried to cut short the period for which a true disrupter typically enters the market as an underdog at first. However, by attempting to improve on the incumbents’ performance and simultaneously miniaturize the form factor drastically, they created a product with no market at all—and a text book case study for how not to disrupt, now widely taught in business school.

There is an important lesson to be learned that many engineers find counter-intuitive: for disruptors, striving for perfection is one of the highest barriers to entry; rather, innovators are advised to go with a Minimum Viable Product first and let the market dynamics shape the product going forward.

It is important to note that while the framework of the Innovator’s Dilemma cannot reliably predict which innovations will disrupt an industry, it neatly points out the weakness of incumbents and how they may get disrupted.

Cloud Databases are the First Real innovation in the Database Field in Decades

There has been no dearth of innovation in the database field over the past four decades but none of the innovations have been able to truly disrupt the database market. For example, database innovations, such as object-orientation, temporal databases, and unstructured data models have all failed to deliver on their original promise to replace the relational database. These innovations ended up becoming features of commercially successful incumbents.

Why did they fail to disrupt? It is because the innovations were actually enhancements of an already very powerful product—faster, more scalable, cheaper, and so forth—but did not innovate along a new dimension.

In contrast, cloud databases are truly radical and novel:
  • They reduce databases to the one singular aspect customers have always appreciated about databases: a query API with a powerful query language.
  • They remove the burdensome and exceedingly expensive aspect of having to operate one’s own database.

To use a metaphor from the consumer space, cloud databases might just become what car sharing has become to buying luxury sedans: Instead of owning an expensive vehicle with regular maintenance requirements, a new generation of buyers is turning to shared cars with pay-as-you-go billing that gives them access to the latest models, configurations, sizes, and capabilities depending on need and occasion, and always ready, washed and gassed up—convenient and economical.

Just like a shared car, cloud databases offer the latest in hardware advances, software upgrades and bug fixes, and fully managed services including maintenance operations. Customers need not bother about backups and other much-loathed activities. And, similar to the benefit of a shared car being guaranteed to be up-to-date on all safety recalls, cloud service providers ensure the latest patches are applied continuously to their systems, thereby minimizing security risks.

How do cloud databases measure up for disruptive potential? In contrast to earlier, incremental but not disruptive innovations, cloud databases appear to have significant potential:
  • Cloud data warehouses disrupt on price: Their pricing, often paired with go-as-you-pay billing, undercuts traditional offerings by an order of magnitude or more.
  • Cloud data warehouses disrupt along new dimensions: Instead of improved speeds-and-feeds, they offer a managed services model. They entered the market decidedly as underdogs regarding performance and scalability but are aggressively catching up with the incumbents.

The above points suggest that cloud data warehouses have the markers of a truly disruptive technology, both from the technology and the market perspective. Naturally, skeptics and detractors continue to point out the lack of maturity of cloud databases on performance and scalability as a barrier to adoption. Ironically, this too is a marker of the disruptive potential of cloud databases.

The enormous investments that cloud data warehouse vendors like Microsoft and have been making over the past decade or so are bearing fruit, including improvements to performance and scalability, which suggests reaching the tipping point is imminent.